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Just-In-Time  concepts  are  of  increasing  importance  in  today’s  competitive  manufactur- 
ing environment.  In  this  context,  we  study  several  discrete  scheduling  problems  as  well  as  a 
production  flow  control  problem. 

The  discrete  scheduling  problems  consider  the  case  where  we  have  a common  due  window. 
Jobs  which  are  completed  within  the  due  window  incur  no  penalty.  Completion  before  the 
due  window  incurs  earliness  penalty  and  completion  after  the  due  window  incurs  tardiness 
penalty.  If  the  location  of  the  due  window  is  a decision  variable,  we  can  provide  a polynomial 
algorithm  for  the  single  machine  case.  For  two  parallel  machines,  we  show  that  the  problem 
is  NP-complete  and  use  dynamic  programming  to  solve  it.  We  present  a heuristic  with 
polynomial  complexity  for  m parallel  machines  and  show  that  the  error  bound  vanishes  as 
the  number  of  jobs  increases.  If  the  location  of  the  due  window  is  a given  parameter,  the 
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problem  is  NP-complete  even  for  the  single  machine  case.  We  use  dynamic  programming  to 
solve  this  problem.  The  main  contribution  of  this  research  is  the  extension  of  the  due-date 
concept  to  the  due-window  concept.  No  prior  research  has  analyzed  earliness  and  tardiness 
performance  measures  in  combination  with  due  windows. 

For  the  production  flow  control  problem , we  consider  a system  consisting  of  one  machine 
for  which  maintenance  is  performed  on  a periodic  basis.  The  time  is  continuous  and  the  time 
horizon  is  finite.  For  each  cycle,  both  the  time  interval  of  operation  and  the  maintenance 
interval  are  deterministic.  During  the  maintenance  interval,  the  system  is  shut  down  and 
cannot  produce.  One  part-type  is  produced  and  the  demand  rate  is  assumed  to  be  constant. 
The  supply  is  assumed  to  be  unlimited.  In  order  to  make  on-time  delivery,  the  objective 
is  to  keep  the  production  as  close  to  the  demand  as  possible.  However,  the  maintenance 
disruptions  make  the  production  deviate  from  the  demand.  We  model  the  problem  as  an 
optimal  control  problem  and  solve  it  by  Pontryagin’s  Minimum  Principle.  We  discuss  both 
the  transient  state  and  the  steady  state.  The  main  contribution  is  the  analysis  of  the 
transient  state. 
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CHAPTER  1 


INTRODUCTION 

In  the  1990s,  international  competition  will  increase  due  to  a number  of  reasons.  One 
of  these  reasons  is  that  advancements  in  information  and  process  technologies  occur  at 
an  accelerated  pace.  Also,  the  consumer  demands  are  increasingly  sophisticated  and  in- 
dividualized. This  results  in  the  need  for  operational  models  which  reflect  long-term  and 
nonfinancial  performance  measures  such  as  customer  satisfaction  with  both  the  product 
and  its  service.  Revolutionary  changes  in  the  management  of  production  systems  are  yet 
another  factor  which  contributes  to  an  increased  competition. 

World-Class-Manufacturing  requires  an  integrated  manufacturing  strategy,  encompass- 
ing Total  Quality  Control  (TQC),  Just-In-Time  (JIT)  Production,  Factory  Automation 
(FA),  and  Total  Productive  Maintenance  (TPM).  In  this  dissertation,  we  will  focus  our 
attention  to  the  Just-In-Time  production  strategy.  According  to  the  Just-In-Time  philoso- 
phy, the  ideal  situation  is  to  to  eliminate  all  inventory.  Inventory  in  its  enlarged  definition 
not  only  refers  to  either  partially  or  fully  completed  products,  but  includes  for  example 
also  excess  amount  of  labor  and  machine  time  which  is  commonly  referred  to  as  idle  time. 
The  Just-In-Time  philosophy  seeks  to  avoid  inventory  of  any  kind.  Another  motivation  to 
reduce  inventory  is  that  inventory  hides  root  causes  of  a problem.  For  example,  inventory 
hides  problems  such  as  rework,  machine  downtime,  change  order,  long  setups,  and  vendor 
delinquencies.  It  is  therefore  desirable  to  identify  and  eliminate  waste  components  such  as 
overproduction,  waiting  time,  transportation,  processing,  inventory,  movement,  and  defec- 
tive products.  These  are  important  steps  toward  a Just-In-Time  environment.  The  process 
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of  creating  a Just-In-Time  production  environment  is  multi-faceted  and  involves  many  con- 
cepts. A very  important  component  in  this  process  is  the  utilization  of  Just-In-Time  oriented 
production  scheduling  techniques. 

At  this  point,  we  note  that  the  area  of  both  sequencing  and  scheduling  is  of  great 
importance  and  is  motivated  by  problems  that  arise  in  production  planning  and  control, 
and  in  general  all  situations  in  which  scarce  resources  have  to  be  allocated  to  activities 
over  time.  General  references  on  the  subject  are  the  classic  book  by  Conway,  Maxwell  and 
Miller  [30],  the  introductory  texts  by  Baker  [17]  and  French  [40],  the  volume  edited  by 
Dempster,  Lenstra,  and  Rinnooy  Kan  [34],  and  survey  papers  such  as  by  Graves  [51]  in  the 
area  of  production  planning,  and  by  Lawler,  Lenstra,  Rinnooy  Kan,  and  Shmoys  [75]  in  the 
general  area  of  Sequencing  and  Scheduling.  Sequencing  and  Scheduling  “is  concerned  with 
the  optimal  allocation  of  scarce  resources  to  activities  over  time''’  ([75]),  and  more  than  any 
other  area  in  operations  research,  the  theory  of  sequencing  and  scheduling  is  characterized 
by  a virtually  unlimited  number  of  problem  types.  In  regard  to  the  deterministic  models, 
the  problems  can  be  classified  according  to  their  complexity.  Complexity  theory  provides 
a mathematical  framework  which  can  classify  problems  as  “easy”  or  as  “hard”  (or  NP- 
complete).  Pioneering  work  in  the  theory  of  NP-completeness  has  been  done  by  Cook  [28] 
and  Karp  [64]  and  [65].  The  text  by  Garey  and  Johnson  [46]  is  a classical  guide  in  this  area. 

For  many  years,  scheduling  research  focused  on  single  performance  measures  that  are 
nondecreasing  in  job  completion  times.  These  are  also  referred  to  as  regular  performance 
measures.  Examples  include  mean  flowtime,  mean  lateness,  number  of  tardy  jobs,  and 
total  tardiness.  Regular  due-date-related  performance  measures  are  unable  to  capture  any 
penalty  for  early  job  completion.  We  thus  are  required  to  use  nonregular  performance 
measures  in  order  to  model  the  objective  of  Just-In-Time  production  scheduling  problems. 
Such  performance  measures  penalize  both  earliness  and  tardiness  where  earliness  occurs 
when  a job  is  completed  before  the  due  date  and  tardiness  is  incurred  when  a job  is  completed 
after  the  due  date. 
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In  this  study,  we  examine  several  production  scheduling  problems  where  both  early  and 
tardy  job  completions  are  penalized.  The  problems  can  be  categorized  in  two  groups:  The 
first  group  consists  of  problems  which  are  modelled  by  discrete  scheduling  models.  The 
second  group  consists  of  problems  which  are  modelled  by  production  flow  control  models. 

For  discrete  scheduling  models,  we  consider  two  problems  in  Chapters  2 and  3.  The  per- 
formance measure  is  due-date  oriented  and  penalizes  both  earliness  and  tardiness.  Accord- 
ing to  the  remark  above,  such  a performance  measure  is  a nonregular  performance  measure. 
More  specifically,  we  consider  a due-date  window  which  consists  of  an  earliest  due-date  and 
a latest  due-date.  There  are  very  few  references  which  allow  for  a zero  penalty  when  the 
completion  time  falls  within  a time  interval  (see  Chapter  2),  and  none  of  them  develop  the 
concept  of  due-window  scheduling.  A survey  of  earliness  and  tardiness  related  literature  is 
given  in  Baker  and  Scudder  [20]  where  the  due-date  related  performance  measures  in  most 
cases  refer  to  the  same  point  in  time  for  the  earliness  penalty  as  well  as  for  the  tardiness 
penalty.  While  this  model  is  applicable  in  many  situations  (for  example,  a production  fine 
where  deviation  from  a point  in  time  affects  the  processing  of  the  part  at  the  next  machine), 
there  are  other  situations  where  there  is  a time  window  during  which  no  cost  is  incurred. 
Consider,  for  example,  the  situation  where  a truck  can  be  loaded  overnight.  So  there  is  a 
time  window  from  the  time  when  the  driver  delivers  the  truck  in  the  evening  to  the  time 
when  the  driver  starts  the  trip  the  next  morning  where  there  is  no  penalty  for  the  products 
which  are  completed  and  loaded  during  the  night  shift.  Products  which  are  completed  early 
require  holding  and  products  which  are  finished  tardy  require  special  shipment,  for  exam- 
ple. Thus,  for  situations  where  the  due-date  as  one  point  in  time  does  not  satisfactorily 
reflect  the  situation,  we  consider  an  interval  of  time  for  each  job  during  which  no  penalty 
is  incurred.  We  refer  to  this  time  window  as  due  window. 

In  the  second  chapter,  we  consider  a single  machine  problem  where  n jobs  have  to  be 
processed.  We  have  a common  due  window  [e,d]  with  earliest  due-date  e and  latest  due- 
date  d which  is  the  same  for  each  job.  For  a completion  before  the  earliest  due-date  e,  an 


4 


earliness  cost  is  incurred.  Completion  after  the  latest  due-date  d results  in  tardiness  cost. 
The  size  of  the  due  window  d — e is  a given  parameter.  Jobs  which  are  completed  within  the 
due  window  do  not  incur  a penalty.  Our  objective  is  to  find  a schedule  which  minimizes  the 
total  penalty.  We  distinguish  between  the  parameter  case,  in  which  the  location  of  the  due 
window  is  given,  and  the  decision  variable  case.  In  the  decision  variable  case,  the  location 
of  the  due  window  is  to  be  determined.  We  do  not  consider  cost  for  due-date  assignment. 

For  the  decision  variable  case  and  a certain  cost  structure,  we  are  able  to  provide  an 
algorithm  which  requires  0(n)  time,  provided  the  indices  are  sorted  according  to  the  shortest 
processing  time  order.  If  the  indices  are  not  sorted,  the  complexity  is  0(nlog(n)).  Since 
even  the  easiest  case  with  unit  weights  is  NP-complete  for  d — e (see  Chapter  2),  there  is 
no  realistic  hope  in  finding  a polynomial  algorithm  for  this  problem.  Thus,  we  use  dynamic 
programming  to  solve  the  parameter  case.  We  observe  that  in  the  case  where  the  size  of 
the  due  window  exceeds  the  largest  processing  time,  we  obtain  a more  efficient  algorithm 
than  for  the  corresponding  problem  for  the  common  due  date  case  d = e.  The  results  of 
this  research  are  to  appear  in  Kramer  and  Lee  [72]. 

In  the  third  chapter,  we  study  the  extension  of  this  problem  to  an  environment  with 
parallel  machines.  Concerning  the  location  of  the  due  window,  we  consider  the  decision 
variable  case.  We  show  that  the  easiest  two-machine  problem  (which  has  unit  cost)  is  NP- 
complete  and  provide  a dynamic  programming  algorithm  to  solve  for  the  optimal  solution. 
Subsequently,  we  present  a heuristic  for  the  two-machine  and  for  the  m-machine  case.  We 
give  an  expression  for  the  upper  bound  of  the  absolute  error  and  show  that  the  relative 
error  vanishes  as  the  number  of  jobs  increases.  The  results  of  this  research  are  to  appear  in 
Kramer  and  Lee  [73].  Chapter  3 concludes  the  problems  with  discrete  scheduling  models. 

We  then  turn  to  the  second  group  of  problems  which  are  modelled  as  production  flow 
control  models.  The  term  production  flow  control  model  includes  two  concepts:  production 
flow  and  control.  By  modelling  the  production  as  a production  flow,  we  model  a discrete 
process  by  a continuous  production  rate.  This  has  the  disadvantage  of  introducing  some 
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inaccuracy  into  the  model  and  it  has  the  advantage  to  simplify  the  analysis.  The  concept  of 
control  concerns  the  control  of  these  production  rates.  When  such  decisions  are  sufficiently 
less  frequent  than  the  event  of  operation  completion,  it  is  reasonable  to  model  a problem  in 
such  a way.  The  problems  which  are  modelled  by  a production  flow  control  model  consider 
part-types  for  which  the  demand  rate  is  assumed  to  be  constant. 

Since  the  early  1980s,  there  has  been  an  increased  interest  in  developing  production 
flow  control  models  for  systems  with  unreliable  machines.  Kimemia  and  Gershwin  [66]  have 
studied  a flexible  manufacturing  system  of  unreliable  machines  where  they  used  a production 
flow  model.  The  complicating  factor  is  the  unreliability  of  the  machines  involved.  These 
break  down  in  a random  fashion  and  the  required  repair  time  is  also  modelled  by  a random 
variable.  Their  main  contribution  is  the  qualitative  characterization  of  an  optimal  control 
structure.  They  introduced  the  notion  of  a hedging  point.  The  hedging  point  is  a production 
surplus  level  which  is  utilized  to  hedge  against  random  machine  failures.  The  control  policy, 
also  referred  to  as  hedging  policy  or  threshold  policy,  operates  as  follows:  There  is  a target 
inventory  level  (referred  to  as  threshold  level  or  hedging  level),  and  the  system  tries  to  reach 
this  level  as  soon  as  possible.  When  the  actual  inventory  is  below  the  target  level,  production 
is  at  full  capacity.  When  the  actual  inventory  is  above  the  target,  the  machine  is  shut  down. 
When  the  target  inventory  level  is  reached,  the  production  rate  equals  the  demand.  The 
target  inventory  level  is  used  to  hedge  against  machine  breakdowns.  Further,  based  on 
assumptions  concerning  the  frequency  of  events  (demand  changes  occur  less  frequent  than 
machine  failures  and  operation  times  are  small  compared  to  the  time  between  failures),  they 
have  proposed  a multi-level  hierarchical  control  algorithm.  The  top  level  is  used  for  long- 
term decisions  such  as  system  configuration,  the  next  level  is  used  to  control  the  production 
flow  and  the  routing.  At  the  lowest  level  of  control  are  part-dispatch  algorithms. 

Subsequently,  the  concept  of  modeling  production  as  continuous  flow,  the  hedging-point 
concept,  and  the  concept  of  a hierarchical  control  algorithm  have  been  refined  and  applied 
to  similiar  problem  settings: 
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Gershwin,  Akella,  and  Choong  [49]  have  improved  the  algorithms  for  the  various  levels 
of  the  hierarchical  control.  Akella,  Choong,  and  Gershwin  [4]  have  studied  the  performance 
of  this  model  for  an  IBM  printed  circuit  card  assembly  line.  They  have  concluded  that 
a hierarchically  structured  policy  such  as  given  in  [49]  is  very  effective  in  scheduling  a 
FMS  and  that  it  can  achieve  high  output  with  low  WIP  and  can  cope  with  changes  and 
disturbances.  Recently,  Bai  and  Gershwin  have  studied  finite  buffer  capacity  models.  In 
[14],  they  have  considered  one-part-type  systems,  multiple-part-type  systems  in  [15],  and 
reentrant  systems  in  [16].  Bai  and  Burhanpurwala  [12]  and  Bai  and  Elhafsi  [13]  have  studied 
unreliable  systems  with  non-negligible  setup  times. 

It  is  difficult  to  obtain  an  optimal  control  for  a system  which  consists  of  unreliable  ma- 
chines. In  the  following  case,  researchers  have  succeeded:  Akella  and  Kumar  [5]  considered 
a manufacturing  system  consisting  of  one  unreliable  machine  which  produces  one  part-type. 
They  analytically  derive  an  exact  solution  and  show  that  the  optimal  control  policy  is  a 
hedging  policy. 

In  Chapter  4,  we  consider  a deterministic  production  flow  control  model  which  can  be 
viewed  as  an  extreme  case  of  a stochastic  model  where  the  variance  of  the  time  between 
failures  and  the  time  between  repairs  is  zero,  i.e.,  we  have  a constant  interval  of  time  for 
the  operation  and  a constant  interval  of  time  set  aside  for  the  maintenance  (repair)  of 
the  machine.  Thus,  the  production  system  under  consideration  consists  of  one  machine  and 
maintenance  is  performed  on  a periodical  (or  cyclical)  basis.  Within  a cycle,  the  operational 
time  is  a constant.  The  time  for  maintenance  is  a constant  as  well.  The  production  system 
produces  one  part-type,  for  which  the  demand  rate  is  assumed  to  be  constant.  We  formulate 
the  problem  as  an  optimal  control  problem.  The  methodology  used  to  solve  the  problem  is 
Pontryagin’s  Minimum  Principle  as  found  in  Pontryagin  et  al.  [86]  or  in  Kirk  [67]. 

We  first  examine  the  structure  of  the  optimal  control  during  the  time  interval  of  oper- 
ation and  we  determine  the  initial  control  level  according  to  the  initial  production  surplus 
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level.  We  then  proceed  to  formulate  and  solve  the  problem  for  one  cycle.  Valuable  in- 
sights are  gained  concerning  the  solution  structure.  The  results  of  the  one-cycle  problem 
are  available  in  Kramer  and  Bai  [70]. 

Next,  we  proceed  to  analyze  the  two-cycle  problem.  Subsequently,  we  solve  the  general 
IV-cycle  problem.  It  is  noteworthy  to  observe  that  the  optimal  control  during  the  first  cycle 
of  the  iV-cycle  problem  for  N > 3 is  identical  to  the  optimal  control  during  the  first  cycle 
of  the  two-cycle  problem.  For  our  problem,  we  define  what  is  understood  by  the  term 
steady  state  and  express  the  number  of  cycles  to  reach  this  state  as  a function  of  the  initial 
production  surplus.  The  results  for  the  problem  with  N > 2 cycles  and  the  discussion  of 
the  steady  state  are  available  in  Kramer  and  Bai  [71]. 


CHAPTER  2 


COMMON  DUE- WINDOW  SCHEDULING 
2.1  Introduction 

Within  the  area  of  production  scheduling,  it  becomes  increasingly  important  not  only 
to  produce  high  quality  products,  but  to  deliver  them  at  the  right  time.  The  right  time 
not  only  requires  to  meet  the  customer’s  due  date,  but  due  to  inventory  costs,  it  also 
requires  to  avoid  earliness.  This  is  reflected  by  the  growing  importance  of  Just-In-Time 
production.  The  performance  measure  which  is  associated  with  the  Just-In-Time  concept 
is  nonregular  (see  Baker  [17]).  A vast  amount  of  literature  has  accumulated  in  the  area  of 
earliness  and  tardiness  penalties,  especially  in  the  context  of  common  due-date  scheduling. 
For  an  excellent  review  of  earliness-tardiness  problems,  see  Baker  and  Scudder  [20].  Hall 
and  Posner  [55]  and  Hall,  Kubiak,  and  Sethi  [56]  provided  the  first  two  pseudopolynomial 
algorithms  to  solve  the  problem.  Related  literature  includes  Lee,  Danusaputro,  and  Lin 
[77],  and  Hoogeveen  and  Van  De  Velde  [58].  Liman  and  Lee  [80]  provided  the  first  finite 
error  bound  for  the  heuristic  method.  Federgruen  and  Mosheiov  [39]  extended  the  objective 
to  certain  nonlinear  objective  functions. 

The  Just-In-Time  Scheduling  model  assumes  that  there  is  only  one  acceptable  comple- 
tion time,  and  any  earliness  or  tardiness  is  penalized.  This  model  fits  intra-manufacturing 
systems  situations  like,  for  example,  a production  line  with  adjacent  machines  processing 
a product  in  sequence.  If  no  buffer  is  used,  starvation  or  blockage  will  occur  when  the 
production  is  not  synchronized.  A buffer  could  be  used  at  additional  cost.  This  type  of 
situation  perfectly  fits  the  traditional  due-date  models  (due  date  as  point-in-time). 

However,  manufacturers  often  consider  a due  date  as  an  interval  rather  than  a point  in 
time.  Thus,  for  each  job  there  are  earliest  and  latest  due  dates.  Any  job  finished  after  its 
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latest  due  date  is  considered  tardy,  and  no  job  can  be  delivered  before  its  earliest  due  date. 
If  it  finished  earlier,  a job  must  be  held  until  its  earliest  due  date  thus  incurring  a holding 
cost.  The  period  between  its  earliest  and  latest  due  date  is  the  due  window.  A job  finished 
within  its  due  window  incurs  no  penalty  (Lee  [76]). 

Anger,  Lee,  and  Martin- Vega  [6]  formally  introduced  the  term  due  window  and  provided 
a polynomial  algorithm  to  minimize  the  number  of  jobs  which  are  completed  outside  their 
due  windows,  where  the  size  of  the  largest  window  is  less  than  or  equal  to  the  smallest 
processing  time. 

Sidney  [102]  considered  earliness  and  tardiness  penalties  with  target  starting  times  and 
due  dates,  and  implicitely  used  the  idea  of  due  windows.  Lakshminarayan  et  al.  [74] 
developed  a more  efficient  procedure  for  the  same  problem.  Cheng  [26]  studied  the  common 
due-window  problem  where  the  size  of  the  window  was  constrained  to  be  less  than  the 
shortest  processing  time,  and  the  penalty  was  calculated  as  the  absolute  deviation  from 
the  middle  of  the  due  window.  Drawing  on  his  practical  experience,  Lee  [76]  has  modeled 
and  solved  several  constant-window  scheduling  problems.  He  has  focused  on  dual  criteria 
scheduling,  differentiating  between  the  cases  where  idle  time  is  allowed  and  where  it  is  not. 

Our  purpose  is  to  extend  the  earliness  and  tardiness  measures  from  the  due-date  case 
to  the  due-window  case.  We  consider  common  due  windows. 

This  chapter  is  organized  in  the  following  way:  The  next  section  introduces  notation 
and  general  properties.  In  Sections  2.3  and  2.4,  we  consider  cases  where  the  due  window  is 
a decision  variable  and  where  the  due  window  is  a parameter,  respectively.  In  Section  2.5, 
we  summarize  the  results  and  comment  on  future  research  directions. 

2.2  Notation  and  General  Properties 

We  assume  all  our  data  to  be  integral.  We  have  n jobs,  and  for  each  job  J3  (j  = 1, . . . , n), 
we  use  the  following  job  attributes: 


1.  Processing  time  pj 
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2.  Starting  time  Sj 

3.  Completion  time  Cj 

The  earliness  penalty  is  a per  unit  of  time  and  the  tardiness  penalty  is  /?  per  unit  of 
time  for  each  job.  We  assume  jobs  are  indexed  in  shortest-processing-time  (SPT)  order,  i.e., 
Pi  < P2  < • • • < Pn-  For  a given  sequence,  [i]  denotes  the  ith  job.  If  a particular  sequence 
of  k jobs  satisfies  > p[2]  > • • • > P[k],  we  say  the  sequence  is  in  longest-processing-time 
(LPT)  order.  We  also  refer  to  the  longest  processing  time  as  pmax , i.e.,  pmax  — Pn- 

A window  is  defined  as  an  interval  [e,d],  where  e is  the  earliest  due  date  and  d is  the 
latest  due  date.  The  due  window  is  common  to  all  jobs,  and  we  consider  its  size  K = d — e 
always  as  a given  parameter  of  the  problem  with  integral  value.  Jobs  which  are  completed 
within  the  due  window  do  not  incur  a penalty.  The  earliness  and  tardiness  of  a job  is  defined 
as  Ej  = (e  -Cj)+  and  Tj  = (Cj  - d)+,  respectively,  where  x+  = max(.T,  0).  The  penalty  of 
job  Jj  is  a function  of  the  completion  time  Cj  and  is  defined  to  be 

a(e  - Cj)+  + /3(Cj  - d)+  = aEj  + /3Tj. 

Our  purpose  is  to  find  a schedule  to  minimize  Z = Y,j(aEi  + PTj)-  We  distinguish 
between  the  parameter  case,  in  which  the  location  of  the  due  window  is  given,  and  the 
decision  variable  case.  In  the  decision  variable  case,  we  do  not  consider  cost  for  due-date 
assignment. 

For  simplification,  we  introduce  partial  sums.  MS  = J2^iPj  represents  the  total 
processing  time  of  all  jobs,  A,  = J2j=iPj  the  total  processing  time  of  jobs  1,...,/.  We 
assume  that  MS  > d — e since  otherwise  we  could  schedule  all  jobs  in  the  due  window  and 
there  would  be  no  problem  to  be  solved.  We  denote  the  cardinality  of  a finite  set  S by  n(S). 

The  following  properties  are  well  known  for  the  common  due  date  problem,  i.e.,  d — e 
(Kanet  [63],  Hall,  Kubiak,  and  Sethi  [56]).  The  first  two  properties  apply  both  to  the 
decision  variable  case  as  well  as  to  the  parameter  case. 
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Property  1:  An  optimal  schedule  exists  without  machine  idle  time  between  the  first  and 
the  last  job. 

Property  2:  In  any  optimal  schedule,  early  jobs  (Ej  > 0)  are  sequenced  in  LPT  order, 
tardy  jobs  ( Tj  > 0)  starting  after  the  common  due  date  ( Sj  > d)  are  sequenced  in  SPT 
order. 

Property  3:  If  d is  a decision  variable,  then  an  optimal  schedule  exists  in  which  Cj  = d 
for  some  j. 

Property  If  d is  a given  parameter,  then  an  optimal  schedule  exists  in  which  Cj  = d 
for  some  j or  the  first  job  starts  at  zero,  i.e.,  .S'jq  = 0. 

We  now  consider  the  extension  of  those  properties  from  the  common  due-date  problem 
( d = e ) to  the  common  due- window  problem  (e  < d).  We  can  see  that  the  first  property  also 
holds  for  the  common  due- window  case.  We  adjust  the  second  property  for  the  common 
due-window  (CDW)  case  as  follows: 

Property  (CDW)  2:  For  every  optimal  schedule,  jobs  completing  before  e ( Cj  < e ) are 
sequenced  in  LPT  order,  and  jobs  starting  after  d (Sj  > d)  are  sequenced  in  SPT  order. 

The  W-jobs  are  not  included  in  Property  (CDW)  2:  We  will  refer  to  them  in  part  (c) 
of  the  following  lemma  which  also  extends  the  third  and  fourth  property  for  the  common 
due-window  case: 

Lemma  1 (a)  For  the  common  due-window  problem,  if  d is  a decision  variable,  an  optimal 

schedule  exists  where  one  job  finishes  at  either  e or  d: 

either  (i)  Cj  = d for  some  j 

or  (ii)  Cj  = d — K = e for  some  j 

(b)  If  d is  a given  parameter,  an  optimal  schedule  exists  where  at  least  one  of  the  following 


conditions  will  hold: 
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either 

(i) 

Cj 

= d 

for  some  j 

or 

(ii) 

Cj  = d 

- K = e 

for  some  j 

or 

(iii) 

5[i] 

= 0 

(c)  V-shape  Property: 

(i)  In  both  parameter  and  decision  variable  cases,  if  there  is  an  optimal  schedule 
with  Cj  = e for  some  j,  then  there  is  also  an  optimal  schedule  with  Cj  = e for 
some  j which  also  satisfies  that  the  E-jobs  are  sequenced  in  LPT-order  and  the 
remaining  jobs  are  sequenced  in  SPT  order.  We  describe  this  schedule  as  having 
a V-shape  around  e. 

(ii)  In  both  parameter  and  decision  variable  cases,  if  there  is  an  optimal  schedule 
with  Cj  = d for  some  j,  then  there  is  also  an  optimal  schedule  with  Cj  = d for 
some  j which  also  satisfies  that  the  T-jobs  are  sequenced  in  SPT-order  and  the 
remaining  jobs  are  sequenced  in  LPT  order.  We  describe  this  schedule  as  having 
a V-shape  around  d. 

(iii)  For  the  parameter  case,  if  there  is  an  optimal  schedule  with  .S'jj]  = 0 and  with  at 
least  one  job  completing  within  the  due  window  (e  < Cj  < d),  then  there  is  an 
optimal  schedule  with  5[i]  = 0 which  also  satisfies  that  all  jobs  with  Sj  < e are 
sequenced  in  LPT  order  and  the  remaining  jobs  are  sequenced  in  SPT  order. 

Proof.  See  Appendix  A.  QED 

2.3  Decision  Variable  Case 

Kanet  [63]  provided  an  0(n*log(n ))  algorithm  to  solve  the  unit  weight  case  (o y = /3j  = 1 
for  all  jobs  Jj ) for  the  common  due  date  ( d = e ) problem.  There  are  many  optimal  solutions 
for  this  problem.  Bagchi,  Sullivan,  and  Chang  [10]  provided  an  algorithm  to  find  all  the 
optimal  solutions.  Lee,  Danusaputro,  and  Lin  [77]  provided  an  0(n2)  algorithm  for  the 
constant-weight  case.  Their  problem  differed  from  Bagchi,  Sullivan,  and  Chang’s  [10]  in 
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that  Lee,  Danusaputro,  and  Lin  included  the  number  of  tardy  jobs  in  the  objective  function. 
We  extend  the  constant-weight  case  to  the  due-window  case  and  provide  an  ()(n  * log(nj) 
algorithm  to  solve  the  problem.  The  main  difference  between  common  due-date  and  common 
due- window  scheduling  is  the  determination  of  the  set  of  jobs  W which  are  completed  within 
the  due  window. 

The  following  definitions  are  used  in  the  decision- variable  case. 

Definition  1 For  a schedule  o,  we  define  the  following  three  sets: 


E(o)  = 

Uj  ■ Cj{<j)  < e} 

(2.1) 

T(o)  = 

{Jj  ■ Sj{c)  > d} 

(2.2) 

W(o)  = 

{Jj  : Cj(o ) > e and  Sj(o)  < d] 

(2.3) 

If  there  is  no  ambiguity,  we  omit  the  parameter  o and  write  E,  T,  and  W instead.  Jobs  in 
E are  called  E-jobs  and  are  completed  at  or  before  the  common  due  window.  Jobs  in  T are 
called  T-jobs  and  start  at  or  after  the  window.  Set  W contains  the  W-jobs.  The  cardinality 
of  these  sets  is  denoted  by  n(E),  n(T),  and  n(W). 

According  to  Equation  (2.3),  the  W-jobs  cover  the  due  window.  Since  we  consider  only 
schedules  o which  have  no  idle  time  during  the  due  window,  J2jew(o)Pj  > d — e holds.  This 
motivates  the  following  definition: 

Definition  2 Define  the  number  nw*  as  follows: 

l 

nw*  = min{/  : y > d — e}  (2.4) 

3= i 

To  express  the  objective  function  in  terms  of  processing  times  (Lemma  3,  below)  and  to 
simplify  the  discussion,  we  introduce  the  concept  of  cumulative  weights.  According  to 
Lemma  1(a),  we  only  need  to  consider  schedules  with  Cj  = e or  Cj  = d for  some  job  Jj. 

Definition  3 We  define  the  cumulative  weight  cw(Jj,o)  for  job  Jj  w.r.t.  schedule  o as 


follows: 
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1.  If  job  Jj  is  the  kth  E-job  in  schedule  o,  i.e.,  Jj  G E(o)  and  [&]  = j,  we  define 

cw(Jj,o)  = (k  - l)a. 


2.  If  job  Jj  is  the  kth  last  T-job  in  schedule  o,  i.e.,  Jj  G T(o)  and  [n  — k -f  1]  = j,  we 
define 

cw(Jj,  o)  - k/3. 


cw 


(Jj,o)  = < 


3.  If  job  Jj  is  a W-job  in  schedule  o,  i.e.,  Jj  G W(cr),  we  define 

( n(E(o)))a  if  Ck  — d for  some  Jk 
(n(T(o))  l)/3  otherwise 

We  also  define 

(n(E(o)))a  if  Ck  — d for  some  Jk 

(n(T(a))  + 1)/?  otherwise 

since  the  cumulative  weight  of  the  W-jobs  is  the  same  for  every  W-job. 


cw(W , cr ) = < 


(2.5) 


Using  the  definitions  for  E,  T , and  W,  we  illustrate  the  two  solution  structures  according 
to  Lemma  1(a)  and  show  the  cumulative  weights  beneath  the  indices  in  Figure  2.1.  These 
will  be  used  in  Lemma  3.  In  Lemma  4,  we  will  show  that  the  W-jobs  consist  of  the  nw* 
smallest  jobs. 


Lemma  2 An  optimal  schedule  o must  satisfy 

max{(n(E(cr))  - l)a,  n(T(a))/3}  < min{n(£((r))a,  (n(T(a))  + 1)0}  (2.6) 

Proof.  According  to  Lemma  1 (a),  we  must  either  have  C3  — e or  Cj  = d for  some  j.  We 
prove  the  Lemma  by  contradiction: 

Assume  that  n{T{c))0  > n(E(cr))a  holds.  Obtain  a schedule  a'  by  moving  the  due 
window  one  unit  to  the  right.  If  Cj  — e for  some  j in  schedule  a,  then  we  have  (whether 
Cj  — d for  some  j or  whether  Cj  ^ d for  all  j ) 

cost(o')  — cost(a)  — n(E(cr))a  - n(T(o))/3  < 0. 
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Figure  2.1.  Two  solution  structures:  Cj  = e and  Cj  = d for  some  j 


If  Cj  7^  e for  all  j in  schedule  a,  then  we  have 

cost(a')  - cost(a)  = ( n(E(a ))  - 1 )a  - n(T(o))/3  < n(E(a))a  - n(T(a))/3  < 0. 

In  both  cases,  schedule  a is  non-optimal,  a contradiction. 

Now  assume  that  (n(E(<r))  — l)a  > (n(T(<r))  + 1 )(3  in  schedule  a.  Obtain  a schedule  a" 
by  moving  the  jobs  one  unit  to  the  right.  The  proof  of  cost(cr")  - cost(a)  < 0 is  symmetric 
to  the  case  above. 

Thus,  an  optimal  schedule  a satisfies  both  n(T(a))l3  < n(E(cr))a  and  (n(E(a))  - l)a  < 
(n(T(a))  + l)/3.  Since  we  also  have  (n(E(a))~  l)a  < n(E(a))a  and  n(T(a))f3  < ( n(T(a))  + 
1 )/3,  Equation  (2.6)  holds.  QED 

Lemma,  3 Let  a be  a schedule  with  no  idle  time  between  jobs  ( C [„]  - 5'p]  = MS)  where 
Cj  = e or  Cj  — d for  some  j.  The  weighted  earliness  and  tardiness  penalty  can  be  expressed 
by  cumulative  weights  for  the  processing  times  as  follows: 

n n 

Y2(.aEj(o)  + PTj(o))  = {^2,cw(Jj,o)*pJ}  - cw(W,o)*(d-  e)  (2.7) 
j= i j= l 

Proof.  According  to  Lemma  1(a),  we  consider  the  two  cases  which  Cj  = e and  Cj  = d 
for  some  j (Figure  2.1).  In  Figure  2.1,  the  E-jobs  are  indexed  e\, . . .,en^,  the  T-jobs  are 
indexed  t\ , . . . , tra(x) , and  the  W-jobs  are  indexed  w\ , . . . , wn( w)  • 
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Now  consider  job  Jej . Each  unit  of  processing  time  of  job  Jtj  contributes  one  unit  of 
earliness  to  each  of  the  jobs  Jei , . . . , JBj_  1 . Thus,  job  JBj  contributes  ((j  — 1 )a)pj  towards 
the  objective  function  which  depends  on  the  number  of  jobs  which  precede  job  J6j  and  on 
the  processing  time  of  job  Jj.  Likewise,  each  unit  of  processing  time  of  job  Jtj  contributes 
one  unit  of  tardiness  to  each  of  the  jobs  Jtl,...,  Jt..  Thus,  the  contribution  of  job  Jt. 
towards  the  objective  function  is  ( jfl)pj ■ 

The  portion  of  W-jobs  exceeding  the  due  window  is  Pj)  — (d  — e).  In  case  of 

Cj  = d for  some  j,  this  portion  would  have  cumulative  weight  n(E)a.  Otherwise  ( Cj  ^ d 
for  all  Jj),  we  have  Cj  = e for  some  job  Jj  and  the  cumulative  weight  is  ( n{T ) + 1)/?. 
According  to  the  definition  of  cw(W,o),  we  can  write: 


n n(E)  n(T) 

Y^iaEjia)  + /3Tj(a))  = (( j - 1)  * a)  * p£j  + J2  U * I3)  * Pt , 

j=i  i= i j=i 

n(W) 

+ cw(W,(r)*(Y2  Pwj  ~ (d- e)). 
j= i 

Using  the  definition  of  cumulative  weights  for  the  jobs,  we  can  write 

n n 

Y2(<*Ej(ct)  + /3Tj(<7))  = {Y2cw(Jj,a)*pj}  - cw(W,(T)*(d- e) 
j= i i=i 


QED 


Lemma,  4 An  optimal  schedule  o exists  wherein  the  set  of  W-jobs  is  the  set  of  the  nw* 
smallest  jobs,  i.e., 

W(o)  = {Jx,...,  Jnw.} 

Proof.  See  Appendix  A.  QED 

Remark  1 In  the  following  algorithm,  we  first  determine  the  number  of  W-jobs  and  then 
sequence  the  E-jobs  and  the  T-jobs.  We  process  the  longest  job  (n)  first  ( with  cumulative 
weight  0),  process  the  second  longest  job  (n  — 1)  at  position  2 or  n,  depending  on  min(a,/3), 
and  so  forth.  In  general,  the  next  job  is  processed  before  or  after  the  due  window  depending 
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on  whether  the  next  early  job  has  a better  cumulative  weight  than  the  next  tardy  job.  Then 
the  W-jobs  are  sequenced,  and  finally,  we  determine  the  location  of  the  due  window. 

Algorithm  1 

1.  Determine  the  number  nw*: 

nw*  = min{/  : Y?j=i  Pj  > d - e} 

2.  Sequence  jobs  n, . . . , nw*  + 1 according  to  their  cumulative  weights: 

[1]  <-  n 

(n(E),n(T), index)  <—  (1,0, n-  1) 

* currently,  n(E)  = 1 E-jobs,  n(T)  = 0 T-jobs 
REPEAT 

* compare  the  next  cumulative  weights  ((n(E)  + 1)  - l)a  and  (n(T)  + 1 )f3 
IF  n(E)a  < ( n(T ) + l)/3 

THEN  n(E)  <—  n(E)  + 1,  [n(£')]  <—  index 
ELSE  n(T)  <—  n(T)  +1,  [n  — n(T)  + 1]  <—  index 

index  <—  index  — 1 
UNTIL  index  = nw* 

3.  Assign  the  position  for  W-jobs: 

WHILE  index  > 0 DO 

IF  (n(E))a  < (n(T)  T 1 )/3 

THEN  [(n  — n(T ) -f  1 — index))  <—  index,  index  <—  index  — 1 
ELSE  [(n(E)  + index)]  <—  index,  index  *—  index  — 1 

4-  C[n  = ptl] 

FOR  i = 2 TO  n DO 

Cb)  = Cb-A  + Pb] 
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5.  Determine  the  location  of  the  due  window: 

IF  (n(E)a  <—  ( n(T ) + l)/3) 

THEN  d = (E(\j]<n(E)+nw*)P\j]),  e = d-K 
ELSE  e = (S([j]<n(£;))P[j'])’  d = e + Ii 

Theorem  1 Algorithm  1 produces  a schedule  which  minimizes  J2j=i(®*Ej  + /3*Tj)  in  linear 
time  (O(n))  provided  the  jobs  are  indexed  in  SPT  order.  If  the  jobs  are  not  indexed  in  SPT 
order,  an  additional  effort  of  0(n  * log(n ))  for  sorting  is  needed. 

Proof.  In  the  first  step,  the  number  nw*  is  calculated  according  to  its  definition.  Ac- 
cording to  Lemma  4,  this  is  also  the  number  of  W-jobs.  At  all  times,  n(E)  is  the  number  of 
E-jobs  and  n(T ) is  the  number  of  T-jobs.  The  E-jobs  and  T-jobs  are  sequenced  according 
to  their  cumulative  weights  in  accordance  with  Lemma  3. 

The  complexity  of  the  algorithm  is  linear  because  each  of  the  five  steps  in  the  algorithm 
is  bounded  by  0{n ) operations.  In  the  case  job  indices  are  not  sorted  in  SPT  order,  we 
need  an  additional  effort  of  order  0(n  * log(n)).  QED 

Remark  2 It  can  be  shown  that  the  above  algorithm  constructs  a schedule  o with  n( E(a))  = 
r (P(n  ~ nw*))/(a  + (3 )]  E-jobs  and  n(T(o))  = n - \{/3(n  - nw*))/(a  + (3)]  - nw*  T-jobs 
where  |V|  = min{/,:  : k > x and  integer}.  Furthermore,  it  can  be  shown  that  for  any  optimal 
schedule,  min (n(E)a,(n(T)  + 1)/?)  is  uniquely  defined. 

2.4  Parameter  Case 

Hall,  Kubiak,  and  Sethi  [56]  used  Dynamic  Programming  to  solve  the  unit-weight  case. 
Since  the  unit  weight  case  is  NP-complete  ([56]),  we  present  a dynamic  programming  algo- 
rithm. Our  contribution  is  to  extend  the  common  due  date  to  the  common  due  window. 
We  provide  a dynamic  programming  algorithm  of  complexity  0(e  * n2  + K * n). 

Remark  3 For  the  due-date  parameter  case,  parameter  d is  non-restrictive  if  d > MS  holds. 
Hence,  it  can  be  treated  like  a decision  variable  case.  Since  this  also  holds  for  the  due-window 
case,  we  assume  here  that  d < MS  holds. 
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According  to  Lemma  1 (b),  an  optimal  schedule  exists  in  which  either  Cj  — d or  Cj  — e 
for  some  j,  or  5[i]  = 0 holds.  Recall  that  A/  is  defined  by  A/  = J2lj= i Pj- 

Algorithm  2a:  Cj  = d for  some  j 

Considering  the  case  Cj  = d for  some  j , we  define  the  recursive  function  for  the  dynamic 
program  as  follows: 

g(7,  t/vr)  is  the  minimal  cost  to  schedule  jobs  1,  . . I such  that 

, ^NT  = J2(j<l  and  C,<d)  Pj 
is  the  total  processing  time  of  non-tardy  jobs. 


Initial  Conditions: 


ff(0,<jvr) 


0 if  fjvr  = 0 
oo  otherwise 


Recursive  Relation: 


9(1,  tNT ) = min  - 


g(l  - 1,  tNT  - pi)  + a * (t^T  ~ Pi  ~ K)+ 
g(l  - l,  tNT)  + P*(Ai  - tNr) 


Answer: 


Z2a  = min  . g(n,tNT) 

(tNT=K,...,e+K) 


Justification 

Lemma  l(c)(ii)  justifies  job  sequencing  from  the  inside  out. 


Complexity 

The  complexity  of  the  Dynamic  Program  is  0(n  * d). 

Algorithm  2b:  Cj  = e for  some  j 

For  the  case  Cj  — e for  some  j , we  define  the  recursive  function  for  the  dynamic  program 


as  follows: 
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h(l,tE ) is  the  minimal  cost  to  schedule  jobs  1,  . . / such  that  tg  - J2(j<i  and  c,<t ) Pj 
is  the  total  processing  time  of  E-jobs. 


Initial  Conditions: 


h(0,tE)  = < 


0 if  tE  = 0 
oo  otherwise 


Recursive  Relation: 


h(l,  tE ) = min 


h(l  - 1 ,<£  - p()  + a * (tE  - pi ) 

h(l  — 1 , t#)  + /3  * ( Ai  — tE  — K)+ 


Answer: 

Z26  = min  MuAe) 

(tE=  0,...,e) 

Justification 

Sequencing  proceeds  in  SPT  order  from  the  inside  out  according  to  Lemma  l(c)(i). 

Complexity 

The  complexity  of  the  Dynamic  Program  is  0(n  * e). 

Algorithms  for  the  Case  S[i]  = 0 

For  this  case,  we  distinguish  whether  a job  crosses  the  due  window,  i.e.,  whether  Sk  < e 
and  Ck  > d for  some  job  J or  not.  If  no  job  crosses  the  due  window,  then  we  must  have 
e < Cj  < d for  some  j.  For  certain  problem  instances,  only  one  of  these  cases  may  apply. 
For  example,  if  pmax  < A , we  will  not  need  Algorithm  2d.  And  if  all  processing  times  are 
multiples  of  10  and  [e,d]  = [24,26],  we  will  not  need  Algorithm  2c. 


Algorithm  2c:  Sp]  = 0 and  e < Cj  < d for  some  j 

In  this  case,  we  define  function  g as  follows,  where  tE  refers  to  the  cumulative  processing 
time  of  jobs  with  Sj  < e: 
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g(l,  tE):  is  the  minimal  cost  to  schedule  jobs  /,  . . . , n such  that  J2(Sj<e  J>/(  pj  = tE 


Initial  Conditions: 


g(n  + l,tE) 


Recursive  Relation: 


0 for  tE  — 0 

oo  otherwise 


Answer: 


g{l,tE)  = min  < 


g{l  + l,tE-pi)  + a*(e-  tE)+ 
g(l  + 1 Ae)  + f3  * {Ai  + tE  — d)+ 


min  g(l,tE). 

( tE=e..d ) 

Justification 

Using  Lemma  1,  parts  (c)(i)  and  (c)(ii),  jobs  with  Sj  < e are  sequenced  in  LPT  order, 
the  remaining  ones  in  SPT  order.  The  cumulative  processing  time  of  jobs  with  Sj  < e,  in 
the  case  where  there  is  at  least  one  on-time  job,  ranges  from  e to  min(d,e  + pmax)-  We 
define  the  recursive  function  by  scheduling  jobs  in  LPT  order  from  the  outside  toward  the 
due  window  by  backward  dynamic  programming. 

Complexity  Analysis 

The  complexity  is  0(n  * d). 


Algorithm  2d:  5p]  — 0,  Sk  < e and  Ck  > d for  some  k 

If  no  job  completes  on-time,  there  must  be  a job  J k which  crosses  the  due  window.  For 
notational  simplification  in  the  dynamic  program,  we  define  A f = A{  + pk  for  i < k and 
A1?  — A{  for  i > k.  For  all  candidate  jobs  ./^  with  p^  > d — e,  we  use  the  following  backward 
dynamic  program: 

hk{l,tE):  is  the  minimal  cost  to  schedule  jobs  /,  . . .,  n except  job  Jk  such  that 

tE=  J2  Pj 

(j>l,  rfk,  Cj<e) 
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Initial  Conditions: 


hk(n  + l,tE)  ~ 


0 for  tE  — 0 

oo  otherwise 


Recursive  Relation: 


hk(l,tE) 


mm 


hk(l  + 1,  tE  - pi)  + a * (e  - tE)  l 7^  k 
hk{l  + l,tE)  + (3  *(Af +tE  — d)  l^k 
hk(l  + 1 ,tE)  l = k 


Answer: 


Z2cl  = min  min  (hk(l,tE)  + /3  * (tE  + pk  - d)). 

k (tE=d-pk,...,e) 


Justification:  The  sequencing  of  E-jobs  is  according  to  Lemma  l(c)(i)  and  of  the  T-jobs 
according  to  Lemma  l(c)(ii).  We  add  the  cost  for  job  Jk  separately  (see  equation  for  Zid ) 
in  order  to  avoid  an  additional  case  distinction  for  hk. 

Complexity  Analysis  for  Algorithm  2d 

According  to  the  state  space,  the  complexity  is  0(n  * e)  for  each  function  hk.  Since  the 
number  of  jobs  for  which  pj  > K holds  is  less  or  equal  to  n,  the  effort  to  evaluate  Z2d  is 
bounded  above  by  0(n2*e). 


The  solution  of  the  ordered  weight  case  is  as  follows: 


Z2  — min(Z2a,  Z2b,  Z2c,  Z2j). 


Complexity  Analysis  and  Justification 

The  individual  complexities  are  0(n  * d),  0(  n*  e),  0(n  * d),  and  0(n2  * e),  respectively, 
resulting  in  0(n2  * e + n * d). 

Remark  4 If  K > pmax,  there  is  always  a job  which  completes  on  time  and  Algorithm  2d  is 
not  needed.  The  complexity  therefore  reduces  in  this  case  (K  > pmax)  to  0(n  * d)  which  is 
more  efficient  than  in  the  common  due-date  case. 
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2.5  Final  Remarks  and  Conclusions 

Since  Just-In-Time  scheduling  is  a special  case  of  Just-In- Window  scheduling,  we  might 
expect  those  scheduling  problems  to  be  more  complex.  However,  for  the  constant  weight 
decision  variable  case,  we  are  still  able  to  provide  a powerful  0(n ) algorithm,  provided  the 
indices  are  sorted  according  to  SPT.  For  the  parameter  case,  a large  due  window  ( K > pmax) 
yields  a more  efficient  algorithm  than  the  corresponding  problem  for  the  common  due  date 
case. 

The  extension  of  this  problem  to  parallel  machines  is  treated  in  Kramer  and  Lee  (1994), 
where  we  show  that  even  the  two  machine  decision  variable  case  is  NP-hard,  and  provide  a 
dynamic  program  for  the  two  machine  case,  as  well  as  a heuristic  with  error  bound  for  the 
general  m machine  case.  Future  directions  include  the  development  of  dynamic  programs 
and  heuristics  with  error  bounds  for  more  general  cost  structures.  The  objective  function 
for  the  decision  variable  case  can  also  be  extended  to  include  a penalty  for  the  assignment 
of  the  location  of  the  due  window. 


CHAPTER  3 

PARALLEL  DUE  WINDOW  SCHEDULING 


3.1  Introduction 

In  this  chapter,  we  are  considering  one  aspect  of  World-Class  Manufacturing.  Its 
pillars  are  Total  Quality  Control  (TQC),  Just-In-Time  (JIT)  Production,  Factory  Automa- 
tion (FA),  and  Total  Productive  Maintenance  (TPM).  One  goal  of  Just-In-Time  Production 
is  to  eliminate  all  inventory.  Inventory  often  binds  significant  portions  of  capital  and  for 
many  companies,  reduction  of  inventory  is  an  existential  problem.  Other  problems  are  also 
addressed  by  the  Just-In-Time  concept.  These  include  rework,  machine  downtime,  change 
order,  long  setups,  vendor  delinquencies,  etc.  Just-In-Time  Production  Scheduling  is  mod- 
eled by  nonregular  performance  measures,  where  both  earliness  and  tardiness  penalties 
are  included.  Many  publications  have  dealt  with  various  earliness  and  tardiness  problems. 
Baker  and  Scudder  give  an  excellent  review  in  [20].  For  the  single  machine  case,  the  first 
pseudopolynomial  algorithms  were  presented  by  Hall  and  Posner  [55]  and  Hall,  Kubiak,  and 
Sethi  [56].  Lee,  Danusaputro,  and  Lin  [77]  provide  a pseudopolynomial  dynamic  program 
for  a more  general  objective  function.  A heuristic  with  finite  error  bound  is  presented  in 
Liman  and  Lee  [80].  Concerning  monotone  penalty  functions  with  respect  to  earliness  and 
tardiness,  Federgruen  and  Mosheiov  [39]  present  optimality  properties. 

This  problem  is  extended  in  two  ways:  First,  the  due  date  formulation  is  generalized 
to  include  the  notion  of  due  windows  where  the  term  due  window  refers  to  a time  interval 
during  which  no  penalty  is  incurred.  Second,  the  results  are  transferred  from  single  machines 
to  parallel  machines. 

Concerning  due  window  scheduling,  Anger,  Lee,  and  Martin- Vega  [6]  consider  a problem 
where  the  objective  is  to  minimize  the  number  of  jobs  completing  outside  their  due  windows, 
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where  the  largest  window  size  is  less  than  or  equal  to  the  smallest  processing  time.  Sidney 
[102]  and  Lakshiminarayan  et  al.  [74]  considered  another  minimax  formulation  where  the 
objective  is  to  minimize  the  maximum  earliness  and  tardiness  penalty.  Restrictions  are 
target  starting  time  and  target  due  date.  The  idea  of  due  window  concepts  was  implicitly 
used.  Cheng  [26]  studies  the  common  due  window  problem  with  the  constraint  that  the 
due  window  is  smaller  than  the  smallest  processing  time.  Unit  weights  with  respect  to  the 
middle  of  the  window  are  used  in  the  objective  function.  Lee  [76]  has  modeled  and  solved 
several  problems  in  the  area  of  constant  window  scheduling.  Kramer  and  Lee  [72]  studied  the 
single  machine  common  due  window  problem  for  constant  weights.  A polynomial  algorithm 
has  been  provided  for  a problem  with  constant  weights. 

The  second  extension  is  from  single  machines  to  parallel  machines.  Emmons  [38]  has 
dealt  with  the  problem  of  constant  earliness  and  tardiness  weights  with  common  due 
date.  As  for  the  one  machine  problem,  the  problem  is  solved  by  assigning  the  jobs  in  non- 
increasing  processing  time  order  to  the  sequence  of  non-decreasing  cumulative  weights. 
In  addition,  Emmons  applied  secondary  criteria  since  many  optimal  solutions  exist  because 
the  cumulative  weights  occur  in  multiples  of  m. 

While  the  extension  from  due  date  to  due  window  preserves  the  polynomial  complexity 
in  the  single  machine  case  for  constant  weights,  this  is  no  longer  true  in  general  for  the 
parallel  machine  case.  For  the  constant  weight  decision  variable  case  with  a common  due 
date,  the  parallel  machine  problem  is  polynomial  [38].  But  the  extension  to  the  due  window 
case  and  parallel  machines  is  NP-hard.  This  holds  even  for  two  machines  and  unit  weights. 

This  chapter  is  organized  as  follows:  In  Section  3.2,  we  introduce  notation  and  review 
general  properties  from  the  single  machine  case  since  they  also  apply  to  the  parallel  machine 
case.  In  Section  3.3,  we  de<d^with  the  complexity  of  the  problem  and  develop  optimality 
properties  for  parallel  machines.  For  the  two  machine  case,  a dynamic  program  and  a 
heuristic  is  presented  in  Section  3.4  and  the  extension  of  the  heuristic  to  an  arbitrary 
number  of  parallel  machines  is  given  in  Section  3.5,  including  an  analysis  of  the  absolute 
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and  relative  error.  We  also  show  that  the  relative  error  vanishes  as  the  number  of  jobs 
increases. 

3.2  Notation  and  General  Properties 

We  assume  that  all  data  used  in  this  chapter  are  integral.  We  have  n jobs,  and  for  each 
job  Jj  (j  — 1, . . . , n),  we  use  the  following  job  attributes: 

1.  Processing  time  pj 

2.  Starting  time  Sj 

3.  Completion  time  Cj 

We  also  define  pmax  — max{pj  : 1 < j < n). 

For  each  job,  a is  the  unit  earliness  penalty  and  (3  is  the  unit  tardiness  penalty,  i.e., 
we  consider  the  constant  weight  case  in  this  chapter.  A window  is  defined  as  an  interval 
[e,d],  where  e is  the  earliest  due  date  and  d is  the  latest  due  date.  The  size  of  the  due 
window  A = d - e is  always  considered  as  a given  parameter  of  the  problem.  However, 
the  location  of  the  due  window  is  considered  as  a decision  variable,  i.e.,  d (e)  is  a decision 
variable.  Jobs  completing  within  the  due  window  do  not  incur  a penalty.  The  earliness  and 
tardiness  of  a job  is  defined  as  Ej  = (e  — Cj)+  and  Tj  = (Cj  — d)+,  respectively,  where 
x+  — max(x,  0).  The  penalty  of  job  Jj  is  a function  of  the  completion  time  Cj  and  is  defined 
as  a(e  — Cj)+  + (3(Cj  — d)+  = aEj  + f3Tj.  The  number  of  parallel  machines  is  denoted  by 
m.  The  ith  machine  is  denoted  by  M‘. 

We  are  given  the  size  of  the  due  window  d — e.  The  due  window  is  common  to  all  jobs. 
We  assume  that  no  costs  are  incurred  in  assigning  the  location  of  the  due  window.  Our 
purpose  is  to  determine  the  location  of  the  due  window  and  to  find  a schedule  to  minimize 

Z = E(«(e  - Cj)+  + (KCj  ~d)+)  = £( aEj  + / 3Tj ). 

i= i j= i 

Throughout  the  chapter,  we  assume  without  loss  of  generality  that  the  jobs  are  indexed 
in  shortest  processing  time  order  (SPT),  i.e.,  pj  < Pj+i  for  all  j.  We  denote  the  processing 
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time  of  the  first  k jobs  by  Ak , i.e.,  Ak  = Ylkj= 1 Pj ■ For  a given  schedule,  [j]1  denotes  the  jth 
job  on  machine  Ml  in  that  schedule. 

For  the  due  window  problem,  we  have  the  following  properties  for  each  machine  [72]: 

Property  1:  There  exists  an  optimal  schedule  without  machine  idle  time  between  the 
first  job  and  the  last  job. 

Property  2:  In  any  optimal  schedule,  jobs  completing  before  e ( Cj  < e)  are  sequenced 
in  reverse  SPT  order,  and  jobs  which  start  after  d ( Sj  > d)  are  sequenced  in  SPT  order. 

Property  3:  There  exists  an  optimal  schedule  where  one  job  finishes  at  either  e or  d: 
either  (i)  Cj  = d for  some  j 

or  (ii)  Cj  = d — K = e for  some  j 

Remark  5 In  order  to  find  an  optimal  schedule,  we  can  restrict  ourselves  to  schedules  which 
satisfy  the  Properties  1-3.  Hence,  throughout  the  entire  chapter,  we  consider  only 
schedules  where  for  each  machine  M\  there  exists  some  job  Jj  such  that  Cj  = e 
or  Cj  = d holds. 

The  main  difference  between  common  due  date  and  common  due  window  scheduling  is 
the  determination  of  the  set  of  jobs  W completing  within  the  due  window.  For  a schedule 
<r,  we  define  the  following  sets  of  jobs: 

E(a)  = {Jj  : Cj(<r)  < e } 

T{o)  = {Jj-.  Sj(cr)  > d) 

W{cj)  = {Jj  : Cj(o)  > e and  Sj(o)  < d} 

WI{cr)  = {Jj  : Sj(c t)  > e and  Cj(cr)  < d} 

Jobs  in  E are  completed  at  or  before  the  window  and  jobs  in  T start  at  or  after  the  window. 
Set  W contains  the  remaining  jobs.  Set  WI  contains  those  jobs  which  are  completely 
processed  within  the  due  window.  According  to  the  job  sets,  we  refer  to  the  jobs  as  E- 
jobs , T-jobs , W-jobs,  or  W I- jobs.  Roughly  speaking,  E-jobs  are  early  and  T-jobs  are  tardy. 
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Strictly  speaking,  the  last  E-job  may  be  completed  at  the  beginning  of  the  due  window  e 
and  the  last  W-job  may  be  tardy.  For  machine  Ml , we  define  JSt(o)  as  follows: 

JS\o)  = {Jj  : Jj  is  scheduled  on  M'} 


Accordingly,  we  define  El(o)  — JS'(a)  D E(o),  Tl(a)  — JS'(o)  fl  T(cr),  W'(o)  = n 

W(o),  and  WP{a)  = JSl{o ) n W{a). 

When  there  is  no  ambiguity  concerning  the  schedule,  we  simply  write  JS\  E\  T\  Wl 
and  WP.  Furthermore,  denote  the  cardinahty  of  a set  S by  n(S),  i.e.,  n(S)  — |5|.  By  the 
above  definitions,  we  have  n(JSl)  — n(El)  + n(Tl)  + n{Wl). 

In  order  to  express  the  objective  function  in  terms  of  processing  times  (compare  [72]), 
and  to  simplify  the  discussion,  we  define  cumulative  weights.  For  the  definition,  please  recall 
that  we  restrict  ourselves  to  schedules  which  satisfy  Properties  1-3.  According  to  Property 
3 , such  schedules  satisfy  either  Cj  = e or  Cj  = d for  some  job  Jj . 

Definition  4 For  a schedule  o,  we  define  the  cumulative  weight  cw(JJ}  o)  for  job  Jj  as 
follows: 

1.  If  job  Jj  is  the  kth  E-job  on  machine  M 1 in  schedule  o,  i.e.,  Jj  € E'(o)  and  [&]*  = j, 
we  define 

cw{Jj,o)  = (k  - l)a. 

2.  If  job  Jj  is  the  kth  last  T-job  on  machine  AT  in  schedule  o,  i.e.,  Jj  € T\o)  and 
[■ n(Ml ) + 1 — k]1  — j,  we  define 

cw(Jj,  o)  = k(3. 


3.  If  job  Jj  is  a W-job  on  machine  Ml  in  schedule  o,  i.e.,  Jj  £ W^o),  we  define 


Cw(Jj,o)  = < 


(n(E\o)))a 
(n(T*(a))  + l )U 


if  Ck  — d for  some  Jk  on  machine  Ml 

otherwise 


29 


We  also  define 


cw(W\  o)  = < 


(n(JE’(cr)))a 

WH«t))+1)/5 


if  Ck  — d for  some  Jk  on  machine  Ml 

otherwise 


since  the  cumulative  weight  of  the  W-jobs  is  the  same  for  every  W-job  in  W\ 


(3.1) 


3.3  Complexity  and  Optimality  Properties 

We  first  demonstrate  that  the  problem  for  the  2-machine  case  is  NP-hard.  As  is  shown 
by  Kramer  and  Lee  [72],  the  constant  weight  decision  variable  case  for  a single  machine  can 
be  extended  from  the  due  date  case  to  the  due  window  case  while  it  remains  polynomially 
solvable.  However,  such  a result  cannot  be  obtained  for  the  parallel  machines  case.  Even 
for  the  unit  weight  case  where  a = /?  = 1,  the  problem  is  NP-hard: 


Theorem  2 The  minimization  of  ( Ej  + Tj)  on  two  parallel  machines  is  NP-hard. 

Proof.  We  reduce  the  Partition  problem  to  our  problem.  An  instance  of  the  Partition 
problem  is  given  by  al5 . . . , and  by  the  question:  Does  there  exist  a subset  I C {1, . . . , N} 
such  that  J2jelai  = ^/2  where  A = J2jLi  aj?- 

e d 


b due  window  *j 

N+  1 

IC{1,...,N} 

N + 2 

{1 

Figure  3.1.  Problem  complexity  for  parallel  machines 


Let  the  window  size  be  d - e = A/2,  the  number  of  jobs  to  be  scheduled  n = N + 2 with 
Pj  — aj  for  j = and  ptv+i  = Pn+2  — A/2  + 1 be  an  instance  of  the  scheduling 

problem.  Figure  3.1  illustrates  the  construction  of  the  scheduling  instance.  Clearly,  a zero 
penalty  solution  for  the  scheduling  problem  exists  if  and  only  if  a partition  exists.  QED 
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(. j-l)c*({j  = l...n(Ei )})  (n(T')  + 1) 

e r 

i 

d jd({j=l...n(Ti)}) 
i 

E* 

Wi 

rpi 

(j-l)a({j  = l...n(Ei)})  n(E')a  jfi  ({j  = 1 . . .n(T*')}) 
Figure  3.2.  Cumulative  weights  for  two  solution  structures 


Please  note  that  if  m,  the  number  of  machines,  is  not  fixed,  the  problem  is  NP-hard  in 
the  strong  sense.  This  can  be  shown  by  reducing  the  problem  3-Partition  to  this  problem. 
In  the  next  lemma,  we  rewrite  the  objective  function  in  terms  of  cumulative  weights. 

Lemma  5 Let  a be  a schedule  which  satisfies  Properties  1-3.  The  weighted  earliness  and 
tardiness  penalty  can  be  expressed  by  cumulative  weights  for  the  processing  times  as  follows: 

n m 

Y^{oiEj(o)  + f3Tj{o))  = ^{{  cw(Jj,cr)*pj}-cw(W',(T)*(d-e)}  (3.2) 

j= i «’= i JjeJSdv) 

Proof.  See  [72]  and  apply  the  result  for  each  machine.  QED 

Remark  6 Figure  3.2  illustrates  the  cumulative  weights  for  the  two  situations  Cj  = e and 
Cj  = d for  some  job  Jj.  Note  that  the  W-jobs  contribute 

mm(a(n(E’)),f3(n(Tl)  + 1))(  P]  - (d-e)) 

J3e.W' 

toward  the  objective  function.  This  explains  why  Wl-jobs  also  have  a cumulative  weight. 

In  the  next  lemma,  we  prove  that  the  number  of  E-jobs  on  any  two  machines  differs  by 
no  more  than  one.  This  is  true  for  all  optimal  schedules.  The  same  statement  also  holds 
for  T-jobs. 
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Lemma  6 For  every  optimal  schedule,  we  have 


maxn(El(o))  — min  n(E\o))  < 1 (3.3) 

Ml  Ml 

maxn(Tl(o))  - min n(Tl(o))  < 1 (3.4) 

Ml  Ml 

Proof.  Assume  that  there  is  an  optimal  schedule  cr  which  satisfies  maX(M(j  n(El(o))  — 
min(Mi)  n(El(o))  > 1.  Let  Ml  and  Mk  be  machines  with  n(E'(o))  = maX(Mi)  n(El{cr))  and 
n(Ek(o))  — min(M()  n(E\o)).  Let  job  J;  be  the  E-job  on  machine  M*  with  cw(Ji,<j)  = 
(n(El(cr))  - l)a.  Obtain  schedule  o'  from  schedule  o by  removing  job  J\  from  machine  Ml 
and  delaying  the  other  E-jobs  of  machine  Ml  by  pi  time  units. 

We  can  disregard  jobs  in  JS\o').  Their  contribution  toward  the  objective  function  is 
either  the  same  or  less,  since  (1)  the  E-jobs  and  T-jobs  of  machine  Ml  have  the  same  cumu- 
lative weights  as  before,  and  (2)  the  W-jobs  of  machine  Ml  satisfy  cw(W{ , o')  - cw(W\  o)  £ 

{0,  — a}  (0,  if  Cj(o)  = e for  some  job  Jj  £ Ex(o)  and  —a  otherwise). 

Job  J[  will  be  processed  on  machine  Mk  instead  as  described  below.  Since  other  jobs 
have  not  been  affected,  we  can  therefore  concentrate  on  the  jobs  JSk(o').  We  distinguish 
the  following  two  cases: 
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1.  Cj  — e for  some  job  Jj  on  machine  Mk. 

In  this  case,  we  schedule  job  J;  on  machine  Mk  such  that  Ci(cr')  = e (see  Figure  3.3, 
part  (a)).  Notice  that  the  only  job  on  machine  Mk  for  which  the  cumulative  weight 
has  changed  is  job  Jp  cw(Jpa')  - cw(Jpa)  = ( n(Ek(a ))  - (n(E\a))  - l))a  < 0. 
Therefore,  we  have 

cost(a')  — cost(a)  < ( cw(Jpo ')  — cw(Jpcr))pi  < 0. 

2.  Cj  ^ e for  all  jobs  Jj  on  machine  Mk . 

According  to  Property  3,  Cj  e for  all  jobs  Jj  on  machine  Mk  implies  that  we  have 
Cj  = d for  some  job  Jj  on  this  machine.  In  this  case,  the  cumulative  weights  of  the 
W-job  for  machine  Mk  may  change.  We  distinguish  the  following  cases: 

(a)  YljJeWk((r)Pj  — (d-  e)  < pi : 

Schedule  job  on  machine  Mk  such  that  Ci(a')  = min(j>gVp(CT/))  Sj(<r)  (see 
Figure  3.3,  part  (b)).  The  jobs  on  machine  Mk  with  modified  cumulative  weight 
are  J\  and  the  set  Wk(cr).  Since  cw(Ji,a ) = (n(El(a))  - l)a,  we  have 

cw(Wk,  a')  — cw{Wk , a)  — a. 

cw(Ji,  a')  — cw(Ji,  cr)  = ( n(Ek(a))a  — ( n(El(a ))  — l)a  < -a. 
Therefore,  we  have 

cost(a')  - cost(cr)  < pi(-a)  + ( pj  - (d  - e))a 

jewk(<r) 

= (pi-(  pj  ~(d~  e)))(-Q)  < ° 

j(zWk(<j) 

(b)  E J}ewk(<r)Pj  ~ (d  — e)  > pp. 

Let  Jw  be  the  job  which  starts  before  the  due  window  in  schedule  a.  Since  we 
have  Cw  ^ e,  we  have  E jjgW{o)  Pj  ~ Pw  < d - e.  In  this  case,  obtain  schedule  a1 
by  scheduling  job  J\  such  that  Ci(a')  = Cw(a)  according  to  Figure  3.3,  part  (c). 
The  cumulative  weights  of  the  former  E-jobs  and  T-jobs  remain  unaltered.  For 
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job  Jw,  we  have  also  cw(Jw,o')  = cw(Jw,o),  since  the  cumulative  weight  of  job 
Jw  depends  only  on  the  number  of  E-jobs  preceeding  it  (compare  Figure  3.2). 
And  this  number  is  the  same  in  both  schedules.  We  have  two  further  cases  to 
distinguish: 

i.  n(E‘ V))  = n(Ek(o))  + 1: 

In  this  case,  job  Jw  is  an  E-job  in  schedule  o'  as  illustrated  in  Figure 
3.3,  part  (c).  For  the  W-jobs  of  machine  Mk  w.r.t.  schedule  o' , we  have 
cw(W,o')  = (r*.(_Efc(<r))  + l)o:.  Further,  cw(W,  o’)  - cw(W,o ) = a > 0, 
< (d~  e),  and  cw(Jho')  < cw(Jho)  imply 

cost(o')  — cost(o) 

< { ~ cw(Jj,o))pj} 

(JjeWH*')) 

-( cw(W , o')  - cw(W , o))(d  — e) 

= «{  S Pj-(d-e)} 

+(cw(Jho')  - cw(Ji,o))pi 

< 0. 

ii.  n(Ek(o'))  = n(Ek(o))\ 

That  is,  the  number  of  E-jobs  on  machine  Mk  is  unchanged.  It  implies  that 
job  Jw  remains  to  be  a W-job  and  job  J/  starts  after  e.  This  case  is  not 
represented  by  Figure  3.3,  part  (c).  Except  job  J/,  all  jobs  in  machine  Mk 
have  the  same  cumulative  weight.  Since  cw(Ji,o)  = ( n(E'(o ))  — l)a,  we 


cost(o')  — cost(o) 

= (cw(Jl,o')  — Cw(Jl,o)pi 
= ( n(Ek(o ))  - (n(E\o))  - 1 ))ap,  < 0. 


have 
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Thus,  in  all  cases,  we  can  find  a better  schedule,  which  contradicts  the  assumption  that 
a is  optimal.  Therefore,  we  must  have  maxM,  n(Ei(o))  - minM,  n{El(a))  < 1.  The  proof 
for  maxM,  n(T\a))  — minMt  n(Tl{cr))  < 1 is  symmetric  to  the  proof  for  maxM,  n(E'(cr))  — 
minM,  n(E'(o- ))  < 1.  QED 

Remark  7 In  order  to  characterize  the  W-jobs,  we  apply  Lemma  4 in  [72]  for  every  machine 
and  have  immediately  that  there  is  an  optimal  schedule  o which  satisfies 

Pj  < Pi  for  all  Jj  £ W\o ) and  for  all  J\  £ El(o)  U T\o)  (3.5) 

for  all  machines  ML  The  following  Lemma  will  extend  this  statement. 

Lemma  7 There  exists  an  optimal  schedule  o*  which  satisfies 

Pj  < Pi  for  all  Jj  £ W{o*)  and  for  all  Ji  £ E(o*)  U T(cr*).  (3.6) 

Proof.  To  facilitate  the  proof,  we  introduce  the  following  definition:  For  a given  schedule 
cr,  define  the  set  WET(o)  as  follows: 

WET(o)  = {(Jj,  Ji)  : Pj  > Pu  JjeW(a)  and  Jt  £ E(a)  U T(o)}  (3.7) 

In  other  words,  WET(o)  is  the  set  of  job-pairs  which  violate  Equation  (3.6).  Therefore,  we 
need  to  prove  that  there  exists  an  optimal  schedule  o*  for  which  n(W ET(o*))  = 0 holds 
where  n(WET(o*))  denotes  the  number  of  pairs  in  WET(cr*).  Let  omin  be  an  optimal 
schedule  which  satisfies 

n(W ET(omin))  = min{n(W ET(o)) : o is  an  optimal  schedule  }. 

If  n(W ET(omm))  = 0,  we  are  done,  i.e.,  there  exists  an  optimal  schedule  which  satisfies 
(3.7). 

Now  assume  that  n(W  ET(omtn))  > 0 and  that  (Jj,Ji)  £ WET  (a*),  i.e.,  Jj  £ W^o*) 
and  J\  £ Ek(o*)  U Tk(o*)  for  some  machines  Ml  and  Mk.  We  show  a contradiction  by 
constructing  an  optimal  schedule  o+  which  satisfies  n(WET(cr+))  < n(WET(amin )),  i.e., 
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which  has  a fewer  number  of  pairs  which  violate  Equation  (3.7).  According  to  the  proof  of 
Lemma  4 in  [72],  we  can  restrict  ourselves  to  the  case  where  schedule  am,n  satisfies  Equation 
(3.5)  for  each  machine.  In  the  following,  we  write  W\  Ek , etc.  for  the  sets  which  refer  to 
schedule  omtn.  We  first  distinguish  between  J/  £ Ek  and  J;  £Tk: 

1.  There  exist  two  jobs  Jj  £ Wx  and  J;  £ Ek  with  pj  > pp. 

We  distinguish  the  following  cases: 

(a)  cw(Ji,amtn ) < a(n(El)  - 1): 

Obtain  schedule  o'  by  interchanging  job  J;  with  the  E-job  on  machine  M'  which 
has  the  same  cumulative  weight.  By  construction,  we  have  cost(a')  = cost(omin). 
Note  that  schedule  o'  violates  Equation  (3.5)  for  machine  M\  Now  observe 
that  by  the  same  interchange  argument  as  in  the  proof  of  Lemma  4 in  [72], 
we  can  construct  a schedule  er+  with  cost(o+)  < cost(o')  and  n(WET(i r+))  < 
n(W ET(omtn)).  This  holds  because  there  are  no  job-pairs  which  violate  Equa- 
tion (3.6)  in  schedule  o+,  but  not  in  schedule  omtn,  i.e.,  we  have  WET(o+)  C 
WET{omin).  And  there  is  at  least  one  job  pair  ((Jj,  J;))  - may  be  more  - which 
violates  Equation  (3.6)  in  schedule  crmm,  but  not  in  schedule  <r+.  That  is,  a+ 
is  an  optimal  schedule  with  fewer  job  pairs  violating  Equation  (3.6)  (since  at 
least  the  pair  ( Jj,  J/)  is  removed  from  the  set  WET(omin)).  This  establishes  the 
desired  contradiction. 

(b)  cw(Ji,amin)  > a(n(El)  - 1): 

In  this  case,  we  further  distinguish  as  follows: 

i.  Cj  = e for  some  job  Jj  on  machine  Ml: 

Obtain  schedule  a'  by  removing  job  J/  from  machine  Mk  and  placing  it 
on  machine  Ml  as  E-job  with  cumulative  weight  a((n(El)  — 1)  + 1)  (< 
cw{Ji,amtn)).  By  construction,  we  have  cost(cr')  < cost(amtn).  Yet  sched- 
ule o'  violates  Equation  (3.5)  for  machine  M1.  Therefore,  we  can  construct 
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an  optimal  schedule  a+  which  satisfies  n(WET(cr+))  < n(W ET(amin)),  a 
contradiction. 

ii.  Cj  7^  e for  all  jobs  Jj  on  machine  Ml: 

For  this  case,  we  further  distinguish  as  follows: 

A-  62jj€W'  Pj ) - Pj+Pi<d-e : 

In  this  case,  we  construct  schedide  a+  by  placing  job  .//  as  W-job  in 
machine  M%  without  changing  the  structure  of  machine  Ml . Since 

cw(Ji,amtn)  > (a((n(El)  - 1)  + 1)  = cw(Ji,o+), 

the  cumulative  weight  of  job  J\  is  either  maintained  or  has  decreased. 
The  jobs  on  machine  Mk  have  either  the  same  or  less  cumulative  weight. 
Therefore,  we  have  a schedule  cr+  with 

cost{o+ ) < cost(a')  and  n(WET(a+))  < n(W ET(amin)). 

That  is,  <r+  is  an  optimal  schedule  with  fewer  job  pairs  violating  Equation 
(3.6). 

B-  CCjjeW'  Pj)  — Pj  + Pi  > d — e: 

In  this  case,  we  finally  apply  Lemma  6 and  conclude  by  n(Ek ) > n(El) 
that  n(Ek)  = n(El)  + 1.  Therefore,  we  have  cw(Jj,crmtn ) = a(n(Ek)  - 
1)  = a(n(Et))  = cw(Ji,amtn).  In  this  case,  we  can  construct  schedule 
<7+  by  exchanging  jobs  J;  and  Jj  without  changing  the  number  of  W- 
jobs,  E-jobs  or  T-jobs  on  any  machine.  Since  both  jobs  have  the  same 
cumulative  weight  in  both  schedules,  we  have  cost(o+)  = cost(omin). 
Since  further  n{W ET(o+))  < n(W ET(crmin)),  we  have  a contradiction 
to  our  assumption. 

2.  There  exist  two  jobs  Jj  £ W’  and  J;  £ Tk  with  pj  > pp. 

This  proof  for  this  case  is  symmetric  to  the  proof  above. 
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QED 

In  order  to  characterize  the  number  of  W-jobs  in  an  optimal  schedule,  we  introduce  the 
quantity  nw. 


Definition  5 Let  nw  be  defined  as  the  maximum  possible  number  of  jobs  which  can  be 
processed  in  time  m * (d  — e ),  i.e., 


max{/  : Ej=i  pj  < m * (d  — e)}  if  px  < m * (d  — e) 


nw  = 


0 otherwise 

Lemma  8 Let  o be  any  schedule  which  satisfies  Properties  1-3  and  Equation  (3.6). 
we  have 


(3.8) 

Then 


n(W(u))  = nw  if  and  only  if  n(W  1(a))  = nw  and  Anw  = m * (d  - e).  (3.9) 

Proof.  By  definition  of  W and  nw , we  have 

TYl  nw 

Y Pj  = Y Y Pj  >m*{d-e)>  Y,pr  (3.10) 

Jj€W(<t)  *=1  Jj£W'(a)  j= 1 

Since  schedule  o satisfies  Equation  (3.6),  the  set  W(o)  consists  of  the  n(W(c r))  smallest 

jobs.  Therefore,  we  have  n{W{(j))  > nw. 

— First  note  that  by  definition  of  W‘  and  by  Property  1,  we  have  Ylj  £W'(o)Pj  — 
d — e for  all  i.  If  n(W(cr))  = nw,  then  the  relations  in  (3.10)  are  equalities  and  therefore, 
J2j]eW(a)Pj  - m*(d  - e)  holds.  This  implies  = d-e  and  lT/!(a)  = W'(o) 

for  all  i.  Therefore,  n(WI(o))  = n(W(o))  = nw  and  Anw  = m*(d-e)  holds. 

Now  note  that  by  definition  of  WI\  we  have  E J,£WP(a)Pi  < d - e for  all  i.  If 
n(WI(o))  = nw  and  Anw  = m * (d  - e)  holds,  we  have  E J^wi(a)Pj  = m * (d  - e). 
This  implies  E Jjewi'(<r)  Pj  = d — e and  W‘(o)  = WIl(o)  for  all  i.  Therefore,  n(lT(cr))  = 
n(WL(o))  = nw  holds.  QED 


Lemma  9 Let  o be  any  schedule  which  satisfies  Properties  1-3  and  Equation  (3.6).  Then 
we  have 


nw  - m < n(WL(o))  < nw. 


(3.11) 
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Proof.  By  definition  of  nw,  we  have  n(WI)  < nw  for  every  schedule  and  thus  n(WI(cr))  < 
nw  holds. 


1.  n(W(cr))  = nw:  If  n(W(a))  = nw  holds,  we  have  n(WI(a ))  = nw  by  Lemma  8. 
Thus,  Equation  (3.11)  is  satisfied. 

2.  n(W(a))  > nw:  Since  schedule  a satisfies  Properties  1 and  3,  we  have  n(Wl(a))  — 
n(WIl{a))  < 1 for  all  i.  Therefore,  we  have  n(W(o))  < n(WI(o ))  + m.  This  implies 
nw  < n(WI(o ))  + m.  Hence,  Equation  (3.11)  holds  as  well. 


3.4  The  Two  Machine  Case 


QED 


In  this  section,  we  consider  the  two  machine  case.  Recall  that  the  single  machine  case 
is  treated  in  polynomial  time.  Kanet  [63]  provided  an  0(n  * log(n))  algorithm  for  due  date 
case,  and  Kramer  and  Lee  [72]  provided  an  0(n  * log(n))  algorithm  for  the  due  window 
case. 

We  use  dynamic  programming  to  determine  the  allocation  of  Wl-jobs  to  the  machines. 
While  Lemma  9 gives  a range  for  the  number  of  Wl-jobs  for  a general  number  of  machines, 
the  following  lemma  refers  to  the  potential  Wl-sets  which  are  of  interest  for  the  Two 

Parallel  Machine  case. 

Lemma  10  There  exists  an  optimal  schedule  o*  for  the  Two  Parallel  Machine  problem 
which  satisfies  Properties  1-3,  Equation  (3.6),  and  the  following  statements  concerning  the 
set  WI(a*): 


1.  If  n(WI(a*))  = nw  - 1,  then  n(W(a*))  - nw  + 1 and  WI(o*)  C {Ji, . . .,  Jnw). 

2.  If  n(WI(a*))  = nw,  then  nw  < n(W(cr*))  < nw  - f 2 and  WI(o*)  C {Ji, . . .,Jnw+\}. 

Concerning  the  E-jobs  and  T-jobs,  we  have  in  case  of  n(W(a*))  > nw,  that  either  E(c r*)  U 
r(a*)  = {Jnw+2,...,Jn}  orE(a*)UT(o*)  = {Jnw+3,...,Jn}  holds. 
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Proof.  Consider  a schedule  o*  which  satisfies  Properties  1-3  and  Equation  (3.6). 

According  to  Equation  (3.11)  in  Lemma  9,  we  have  nw  - 2 < n(WI(o*))  < nw.  Ac- 
cordingly, we  distinguish  the  cases  n(WI(o*))  - nw  - 1 and  n(WI{o*))  - nw. 

1.  n(WI(o*))  — nw  - 1: 

In  this  case,  we  have  n(W{o*))  < nw  + 1 by  Property  3.  and  n(W(o*))  ± nw  by  (3.9). 
Hence,  we  have  n{W(o*))  = nw  + 1 and  by  Equation  (3.6)  W(o*)  = {Ji,.  ..,Jnw+ 1}- 
Hence,  there  are  two  jobs  which  are  W-jobs,  but  not  Wl-jobs.  By  interchange  argu- 
ment we  can  determine  one  of  them:  Job  Jnw+\  can  always  be  scheduled  as  non-  Wl-job 
without  changing  the  cost  and  without  changing  the  number  of  Wl-jobs  or  W-jobs. 
Therefore,  there  exists  an  optimal  schedule  o'  which  satisfies  Properties  1-3,  Equation 
(3.6),  n{W{o'))  - nw  + 1,  n{WI{o'))  - nw  - 1,  and  WI(o')  C { Ju  . . . , Jnw}. 

2.  n{WI{o*))  = nw: 

(a)  Anw  = 2 * (d  - e): 

By  (3.9),  we  have  n(W(o*))  = nw,  WI(o*)  = {Jt, . . .,  Jnw},  and 

{J\,  ... , Jnw } C { J\ , • • • , J\ mo+l } . 

(b)  Anw  <2  *(d-e): 

Since  Y,(jj£W(a*))  Pj  > 2 * (d  - e)  holds  and  since  schedule  o * satisfies  Equation 
(3.6),  we  have  n{W{o *))  > nw  + 1.  Since  schedule  o*  satisfies  Properties  1-3,  we 
have  by  the  definitions  of  W and  WI  that  n(W(o*))  < nw  + 2 holds.  Therefore, 
we  have  n(W(o*))  6 {nw  -f  l,nw  + 2}. 

i.  In  case  of  n{W(o*))  = nw  + 1,  WI(o*)  C {Ji,...,Jnw+ 1}  holds. 

ii.  In  case  of  n(W  (o*))  = nw+2,  there  are  two  jobs  which  are  W-jobs,  but  which 
are  not  Wl-jobs.  By  interchange  argument  we  can  determine  one  of  them: 
Job  Jnw+ 2 can  always  be  scheduled  as  non-  Wl-job  without  changing  the  cost. 
Therefore,  in  case  of  nw+2  W-jobs,  there  exists  an  optimal  schedule  o'  which 
satisfies  Properties  1-3,  Equation  (3.6),  and  WI(o')  C {J\, . . . , Jnw+\}. 
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Now  consider  the  E-jobs  and  T-jobs.  If  n(TE(cr*))  > nw,  we  have  that  either  n(W(o*))  = 
nw  + 1 or  n(  W (cr*))  = nw  + 2 holds.  If  n(W(o*))  — nw  + 1,  we  have  by  Equation  (3.6)  that 
E(a*)  U T(o*)  = { Jnw+2,  ■ ■ ■ , Jn}  holds.  And  if  n{W(o*))  = nw  + 2,  we  have  by  Equation 
(3.6)  that  E(a*)  U T(a*)  = { Jnw+3,  ...,Jn}  holds.  QED 

Dynamic  Program 

According  to  the  previous  lemma,  we  must  examine  the  cases 

1.  n(WI)  = nw  - 1 and  WI  C {Ji, . . .,  Jnw}- 

In  this  case,  we  have  nw  different  sets  for  the  Wl-jobs.  We  obtain  those  sets  by 
selecting  the  job  Jk  which  is  not  a Wl-job.  Then,  we  have  WI  = { Jx,  . . . , Jnw}  - { Jk }. 
This  is  done  for  each  Jk  € { Ji, . . . , Jnw}. 

2.  n(WI)  = nw  and  WI  C {Ji, . . . , Jnw+1}. 

In  the  same  manner,  we  select  a job  Jk  6 {J\,.  ..,Jnw+i}  which  is  not  a Wl-job. 
Hence,  the  remaining  jobs  {Ji, . . .,Jnw+i}  - {Jk}  are  the  Wl-jobs.  Again,  this  is 
done  for  every  job  Jk  e {J\, . ..,Jnw+1}. 

The  following  dynamic  program  is  used  to  determine  whether  the  potential  Wl-sets  can 
indeed  be  scheduled  within  the  window.  The  function  depends  on  job  Jk , index 

/ (<  nw  + 1),  and  parameter  t.  The  function  determines  whether  jobs  Ji, . . . , J;,  except  job 
Jk  {k  G {1, . . .,  nw  + 1})  can  be  scheduled  as  Wl-jobs  such  that  the  cumulative  processing 
time  for  machine  M1  equals  a given  parameter  t,  i.e.,  Pi  = 

Algorithm  1:  Feasibility  of  Wl-jobs 

fk(l,t ) = Y ES  if  there  is  a subset  of  jobs  S C {Ji, . . -,Ji}~  {Jk}  such  that  J2jesPj  = t 
and  Y}j= i,  jz£k  Pj  — t < d — e and  fk(l,  t ) = NO  otherwise. 

The  jobs  in  S correspond  to  the  ones  which  are  scheduled  in  the  due  window  of  machine  M1. 
In  determining  the  optimal  schedule  for  the  2 machine  problem,  we  are  interested  in  the 


41 


function  values  fk(nw,  t)  for  all  k 6 {l,...,raiy}  and  all  t 6 {(Anw-pk-(d-e))+,...,d-e} 
and  fk(nw  + 1,  t ) for  all  k 6 {1, . . . , nw  + 1}  and  all  t € {(.Am4'+1  — pk  — (d  — e))+, . . ,,d  — e}. 

These  values  of  t will  guarantee  that  a YES  answer  can  only  occur  when  Ylj^wi1  Pj  — 
d — e holds  for  both  machines  i € {1, 2}.  Note  that  these  values  of  t cover  the  range  of  the 
total  processing  times  of  Wl-jobs  in  machine  M1. 


Initial  Conditions: 


fk(0,t)  = 


YES  t = 0 
NO  otherwise 


Recursive  Relation: 


/*(/-!,<)  if  l = k 


YES  if  / / k and  /*(/-  l,t~Pi)  = YES 

YES  if  l^k  and  fk(l  - 1 ,t)  = YES  and  Pj  ~ t < d - e 

NO  otherwise 


where  1 = 0,...,  nw  -f  1 and  t = 0, . . . , d — e. 


Answer: 

The  results  of  this  dynamic  program  are  used  in  various  places.  As  mentioned  above,  we 
are  interested  in  fk(nw,  t)  for  all  k 6 {1, . . . , nw}  and  all  t e {(Anw -pk-(d-e))+ , . . ,,d-e} 
and  fk(nw  + l,t)  for  all  k e {1, . . . , nw  + 1}  and  all  t € {(Anw+1  -pk  - (d  - e))+, . . ,,d-e}. 

The  recursive  relation  skips  index  k (since  job  Jk  is  not  a Wl-job.  At  stage  l (/  ^ k ), 
job  J\  is  either  placed  in  machine  M 1 or  M2. 

Outline  for  Algorithm  2: 

The  algorithm  for  the  two  parallel  machine  case  is  given  in  Algorithm  2 in  Appendix  B. 
Conceptually,  we  proceed  as  follows: 


1.  We  first  calculate  cumulative  weights  for  E-jobs  and  T-jobs  according  to  the  a and  (3 
values.  The  cumulative  weights  (0, 0,  a,  a,  2a,  2a, . . . , /3,  (3, 2 (3, 2(3 , . . .)  are  generated 
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in  order  of  increasing  value,  but  are  indexed  from  n in  decreasing  order:  cwn  = 0, 
i = 0,  ctiin_ 2 = min(a,/3),  etc.  until  cwnw.  These  cumulative  weights  are 
assigned  to  the  corresponding  E-jobs  and  T-jobs.  For  the  W-jobs,  we  either  need  to 
use  cwnw+ 2 and  cwnw+\  or  cwnw+i  and  cwnw,  as  will  be  shown  below.  Recall  that 
according  to  Lemma  9,  there  exists  an  optimal  schedule  where  the  number  of  IVI-jobs 
is  either  nw  — 1 or  nw. 

If  the  number  of  Wl-jobs  is  nw  — 1,  then  we  have  by  Lemma  10  that  n(W)  — nw  + 1 
and  furthermore,  E U T = {Jnw+ 2>  • • • , Jn}  holds.  The  contribution  of  the  E-jobs  and 
T-jobs  to  the  objective  function  is 

n 

CW]  * Pj. 

j=nu>+2 

If  the  number  of  Wl-jobs  is  nw,  we  have  n{W)  < nw  + 2 and  furthermore  that 
E liT  — {Jnw+ 3, . . . , Jn}  holds.  Their  contribution  to  the  objective  function  is 

n 

£ CWj*Pj. 
j=nw- f3 

We  examine  each  schedule  with  nw  Wl-jobs  in  Step  2 and  each  schedule  with  nw  - 1 
Wl-jobs  in  Step  3.  Among  these  schedules,  there  exists  one  which  has  the  optimal 
objective  value. 

2.  We  use  the  Dynamic  Program  to  examine  all  schedules  which  have  nw  Wl-jobs.  Ac- 
cording to  Lemma  10,  the  Wl-jobs  satisfy  WI  C {-h, . . . , Jnw+1}.  Due  to  the  num- 
ber of  Wl-jobs , we  need  to  take  out  one  job.  Accordingly,  for  each  job  Jk  where 
k € {1, . . . , nw  + 1},  we  need  to  consider  WI  = { Jx, . . . , Jnw+1}  - {J*J.  Among  the 
non-  Wl-jobs,  jobs  Jk  and  Jnw+ 2 are  the  next  two  smallest  jobs  and  will  be  scheduled 
next  to  the  Wl-jobs.  Due  to  symmetry,  we  only  need  to  examine  those  situations 
where 

Pj  < I]  Pj 
JjeWVnVu...,^}  Jj€WPn{Jlt...,Jnw} 

We  examine  both  cases  for  jobs  Jk  and  Jnw+2  '■ 


43 


(a)  Job  Jk  is  scheduled  on  machine  M 1 and  Job  Jnw+ 2 is  scheduled  on  machine  M 2: 
Jk  € JS1  and  Jnw+ 2 G J52 

(b)  Job  Jnw+2  is  scheduled  on  machine  M 1 and  Job  J k is  scheduled  on  machine  M 2: 
Jnw+ 2 6 JS1  and  Jk  G J S2 

Please  note  that  we  do  not  need  to  consider  the  cases 

(a)  Jk  G JS1  and  Jnw+ 2 € JS1  or 

(b)  J nw+2  G and  Jk  G J52  . 

Subsequently,  we  calculate  the  associated  cost  and  keep 

(a)  the  minimum  objective  value 

(b)  the  cumulative  processing  of  the  Wl-jobs  in  machine  M 1 

(c)  the  machine  to  which  job  Jk  is  allocated 

in  order  to  be  able  to  reconstruct  the  schedule. 

3.  Next,  we  examine  all  schedules  with  nw  — 1 Wl-jobs  similiarly  to  the  situation  in  Step 
2 and  according  to  Lemma  10. 

4.  If  there  is  an  optimal  solution  with  nw  Wl-jobs , we  calculate  the  location  of  the 
common  due  window  such  that  the  starting  times  for  all  jobs  are  nonnegative  and 
either  Sn  = 0 or  Sn-\  = 0 holds. 

5.  This  step  is  similiar  to  Step  4 and  concerns  the  case  that  the  optimal  solution  has 
nw  — 1 Wl-jobs. 

Theorem  3 Algorithm  2produces  a schedule  which  minimizes  Y^=  i(a  * Ej  + 0*  Tj)  in  time 
0(n2  * (d  - e)). 

Proof.  The  steps  are  according  to  the  outline.  According  to  Lemma  9,  we  know  that 
there  is  an  optimal  schedule  o which  satisfies  nw  - 2 < n(WI(o*))  < nw.  According  to 
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machine  M 1 


ei  d\ 


1 

n(E') 

nt\ 

n1 

e2  d2 

J L 


machine  M2 

1 

n(E2) 

nt2 

n2 

Figure  3.4.  Positions  of  the  E-jobs  and  T-jobs  where  nti  = n±  — n(Tl ) -f  1 

Lemma  10,  the  algorithm  considers  all  the  sets  WI  which  are  necessary  to  find  an  optimal 
solution.  For  each  of  those  sets,  the  algorithm  examines  all  possible  workloads  for  machine 
M 1 due  to  Wl-jobs  (and  thereby  also  all  possible  workloads  for  machine  M2  due  to  WI- 
jobs).  According  to  Lemma  7,  the  contribution  of  jobs  Ju  . . . , Jn(wi)+ 2 toward  the  objective 
function  value  is  calculated.  And  according  to  Lemma  5,  the  total  cost  is  obtained  by  adding 
the  cost  for  jobs  {Jn(wi)+3i  ■ • - ,Jn}  according  to  Step  1. 

The  location  of  the  due  window  is  determined  in  such  a way  that  all  starting  times  are 
nonnegative  and  one  of  the  jobs  is  processed  starting  at  0. 

The  positioning  of  the  E-jobs  and  T-jobs  is  illustrated  in  Figure  3.4. 

Concerning  the  time  complexity,  consider  the  following:  Step  1 is  of  linear  complexity 
in  time,  since  n - ( nw  - 1)  cumulative  weights  are  determined  and  the  contribution  of  jobs 
Jnw+2,  ■ ■ ■,  Jn  or  Jnw+3,. . ,,Jn  requires  a linear  time  effort.  The  calculation  of  Anw  and 
Anw~l  also  involves  linear  effort. 

The  results  of  the  dynamic  program  are  utilized  in  Steps  2 and  3 and  contribute  an  effort 
of  order  0(n 2 * (d  — e)).  This  is  due  to  the  fact  that  nw  + 1 different  dynamic  programs  are 
used  (k  = 1, . . . , nw  + 1)  and  that  the  state  space  is  (nw  + 1)  * (d  — e).  In  Step  2,  we  have 
a FOR  loop  for  each  k £ {1, . . . , nw  + 1}.  Note  that  in  the  FOR  loop  for  t , the  range  for 
time  t is  Upper  — Lower  + 1 < d — e.  We  have  a constant  number  of  operations  for  each 
value  of  t.  The  total  effort  for  Step  2 and  Step  3 is  therefore  bounded  by  0(n 2 * (d  — e)). 
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The  Steps  4 and  5 are  of  linear  complexity,  since  the  summation  of  PE1  and  PT ‘ for 
i G {1,2}  involves  at  most  0(4  * n)  operations,  the  determination  of  the  starting  times  has 
also  linear  effort  and  the  remaining  parts  have  constant  effort. 

Therefore,  the  overall  complexity  is  determined  by  Algorithm  1 and  is  0(n2  * (d  - e)). 

QED 

Heuristic  for  Two  Machine  Case 

In  the  following,  we  present  a heuristic  with  linear  time  complexity  and  provide  an 
expression  for  the  absolute  error  bound.  The  heuristic  is  based  on  a greedy  strategy  to 
place  as  many  jobs  within  each  window  as  possible.  Recall  that  the  pseudopolynomial  time 
complexity  in  Algorithm  2 stems  from  the  determination  of  the  maximal  number  of  W-jobs 
using  Algorithm  1. 


Algorithm  3:  Heuristic 


Step  1.  Same  as  in  Algorithm  2 

Step  2.-5.  Apply  a greedy  strategy: 

n(WI1)  = max{l  : £-=1  pj  < (d  - e)}} 
n(WP)  = max{l  : ft  <(i-  .)}} 

n(WI)  = n(WIl)  + n{WP) 


Do  all  the  calculations  as  in  Steps  2.-5.  for  the  case  n(WI)  Wl-jobs  and 


Pi- 


Note  that  the  heuristic  generates  a schedule  which  satisfies  Properties  1-3.  In  the  fol- 
lowing, we  first  provide  an  upper  bound  for  the  difference  of  two  consecutive  cumulative 
weights. 

Lemma  11  Let  {cwn_2*i  ■ 0 < i < [(n  — nw)/ 2J}  be  the  [(n  — nw)/2  + 1J  lowest  numbers 
of  the  set  {k  * a : k > 0 and  k is  integer  } U {k  * ft  : k > 0 and  k is  integer  } such  that 
cwm  < cwm—2  holds  for  m = n-2*([(n-  nw)/2\  -f  1 ),...,  n.  In  addition,  let  cwn_2*i- 1 = 
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cwn-2*i  for  i = 0, . . |_(«  — nw)/ 2J  — 1.  In  other  words,  these  weights  cwn, . . .,cwnw  are 
identical  to  the  ones  which  are  generated  in  Step  1 of  Algorithm  2. 

Then  we  have 

0 < cwm  — < min{a,/f}  for  nw  < m < n — 1. 

Proof.  By  definition,  we  have  0 < cwm  - cwm+l  for  nw  < m < n - 1.  Now  assume 
that  cwm  — cwm+i  > min{a,/?}  holds.  Let  7 = min{a,/3}  and  define  cw  = min{A;  * 7 : 
k * 7 > cwm+ 1}.  By  definition  of  cw,  we  have  cw  - 7 < cwm+i  and  by  cwm+ 1 + 7 < cwm , 
we  have  cw  < cwm.  Since  cw  belongs  to  the  set  {k  * a : k > 0 and  k is  integer  } U {k  * fd  : 
k > 0 and  k is  integer  },  this  provides  a contradiction  to  the  definition  of  the  numbers 
cwn,...,cwnw.  QED 

Lemma  12  The  Complexity  of  Algorithm  3 is  linear  (0(n)),  provided  that  the  jobs  are 
indexed  in  SPT  order.  Otherwise,  the  complexity  is  0(n  * log(n)). 

The  absolute  error  is  at  most  mm(pnw+2,d  - e)  * min (a,/3)  for  nw  + 2 < n and  0 
otherwise. 

Proof.  Concerning  the  complexity,  note  that  the  W-jobs  are  heuristically  determined  in 
linear  time.  Since  only  one  case  for  n(WI ) and  only  one  value  for  the  workload  of  117- 
jo  6. s on  machine  M 1 is  considered,  the  heuristic  requires  an  effort  which  is  linear  in  time. 
Further,  sorting  of  the  indices  results  in  a total  effort  of  0(n  * log(n)). 

Concerning  the  absolute  error,  note  the  following:  According  to  the  definition  of  nw, 
the  heuristic  will  find  at  least  nw  — 1 Wl-jobs.  The  heuristic  will  therefore  obtain  0 cost 
when  the  number  of  jobs  satisfies  n < nw  + 1.  In  the  following,  we  therefore  only  need  to 
consider  the  case  n > nw  -f  2.  Also,  the  heuristic  will  produce  a schedule  which  satisfies 
Properties  1-3  and  Equation  (3.6).  Furthermore,  the  E-jobs  and  T-jobs  will  be  optimally 
scheduled  according  to  the  cumulative  weights. 

According  to  Lemma  9,  every  optimal  schedule  o*  which  satisfies  Equation  (3.6)  also 
satisfies  Equation  (3.11),  i.e.,  nw  — 1 < n(W I(o*))  < nw  for  this  two  machine  case.  In 
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order  to  compare  the  cost  of  the  schedule  obtained  by  the  heuristic  to  the  optimal  objective 
value,  we  first  consider  two  schedules  op  and  op  which  satisfy  Properties  1-3 , Equation 
(3.6),  and  further  n{WI(ao))  = nw  and  n(WI((J\))  = nw  — 1.  Furthermore,  we  require 
that  for  both  schedules,  the  E-jobs  and  T-jobs  are  optimally  scheduled  according  to  their 
cumulative  weights,  i.e.,  if  we  have  Jj,  J;  £ E(<Ji)  U T(ot)  and  pj  < pu  then  we  must  have 
cw(Jj,Oi)  > cw(Ji,Oi ) for  i = 1,2. 

According  to  the  different  cases  in  Lemma  10,  we  can  write  the  objective  function  value 
of  schedule  <7o  in  each  case  where  n(WI)  = nw  holds  as  follows 

cost(c j0)  = cwnw+1  * (pk0  + Ti  - (d  - e))  + cwnw+2  * (pnw+2  + T2  - (d  - e)) 

n 

+ CWJ  * Pi 

j=nw+ 3 

where  kO  £ {1, . . .,nw  + 1},  Tj  + T2  + pk0  = A™*1,  and  Tx  < T2  < d - e hold.  Due 
to  optimality  and  cwnw+i  > cwnw+2,  we  can  restrict  ourselves  to  the  case  that  Tx  < T2 
holds.  The  expression  above  represents  the  case  that  WI{gq)  = {Jx, . . . , Jnw+1}  - {J^o}, 
'Lj]£WI"(<t0)Pj  = T\i  and  E J,evy/‘2(a0)  ft  = r2  (>  Tx)  hold  where  {*1,  *2}  = {1,2}.  Due 
to  optimality,  cwnw+ 1 > cwnw+2,  and  pk0  < pnw+ 2,  we  only  need  to  consider  the  case  that 
job  J kQ  is  scheduled  on  the  same  machine  which  has  a load  of  Tj  processing  time  units  due 
to  Wl-jobs. 

Likewise,  according  to  the  different  cases  in  Lemma  10,  we  can  write  the  objective 
function  value  of  schedule  <Ti  in  each  case  where  n(WI)  = nw  - 1 holds  as  follows 

cost(a i ) = cwnw  * (pk\  + T3-(d-e))  + cwnw+1  * ( pnw+i  + T4  - (d  - e)) 

n 

+ CWJ  * Pi 

j=nw+ 2 

where  kl  £ {1, . . nw},  T3  + T4  + pkl  = Anw , and  T3<TA<d-  e hold. 

According  to  the  generation  of  the  cumulative  weights  (which  is  independent  of  schedule 
cq  or  <ti),  we  have  either  cw^-^j  — cwnw4.\  or  cwnw4.\  — cWjiw4.2\ 


48 


1.  cwnw  — cwnw. |_i  ; 

In  this  case,  we  have 

cost(a  o)  — cos<(cti) 

= {cwnw+ 1 * (pk o 4 -Ti-(d-  e))  + cwnw+2  * ( pnw+2  + T2-(d-  e))} 

- {cWnw+\  * (Anw+1  -2  *(d-e))  + cwnw+2  * pnu,+2} 

= cwnw+i  * {{d  — e)  — (Anw+1  — pk o — Ti))  + cwnw+2  * (T2  — (d  — e)) 

= (Kcwnw+ 1 — cwnw+2)  * ((d  — e)  — T2)) 

Please  note  that  0 < ((d  - e)  - T2)  < min((d  - e),pnw+2 ) holds  due  to  d - e > T2, 
(d  - e)  - T2  < (d  - e ) and  T2  + pnw+2  > (d  - e)  (i.e.,  (d  - e)  - T2  < pnu,+2).  Using 
Lemma  11,  we  conclude 

0 < cost(a0)  - cost(ai)  < (min(a,/?))  * min((d  - e),pnw+2). 

2.  cwnw+\  = cwnw+2: 

In  this  case,  we  have 

cost(a0)  - cost  (a 1 ) = ( cwnw+1  - cwnw ) * (pkl  + T3  - (d  - e)) 

Note  that  cost(a0)  — cost(ai)  < 0,  since  we  have  ( cwnw+i  - cwnw ) < 0 and  (pk  1 + 
T3-(d-  e))  > 0.  Similiarly,  we  have  0 < pkl  + T3  - (d  - e)  < pkl  < pnw+2  due  to 
Pkl  + T3  > d — e and  T3  < d — e.  Since  schedule  00  has  nw  Wl-jobs,  we  must  have 
Pnw  < d — e.  Since  kl  < nw  holds,  we  conclude  pki  < d - e.  Using  Lemma  11,  we 
finally  have 

-(min(a,/?))  * min((d  - e),pnw+2)  < cost(o0)  - cost(oi)  < 0 
We  can  write  for  both  cases  ( cwnw  = cwnw+1  and  cwnw+1  = cwnw+2 ) 

\cost((T0)  - cost(oi)\  < (min(a, /?))  * min((d  — e),pnw+2). 
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Now,  if  the  heuristic  and  the  optimal  schedule  have  a different  number  of  Wl-jobs,  we 
have  by  the  above  considerations  that  the  absolute  error  bound  is  (min(a,/?))  * min((d  - 
e),Pnw+ 2)-  If  the  heuristic  and  the  optimal  schedule  have  the  same  number  of  Wl-jobs, 
the  same  cumulative  weight  will  be  applied  to  jobs  Jn(wl)+2,  ■ • • , Jn-  Only  due  to  the 
situation  where  there  are  2 W-jobs  which  are  not  Wl-jobs , there  could  be  a difference  due 
to  the  smaller  of  these  two  jobs.  Again,  an  upper  bound  for  this  difference  is  (min(a,/?))  * 
min((d  — e),  pntu+2). 

Therefore,  in  all  cases,  the  absolute  error  is  at  most  (min(a,/3))  * min((d  — e),pnw+2). 

QED 


3.5  Generalization  for  the  m machine  case 

According  to  Lemma  5,  the  cumulative  weight  of  E-jobs  and  T-jobs  is  easily  determined 
by  generating  the  cumulative  weights  in  ascending  order  and  assigning  the  jobs  in  LPT 
order  to  them.  Thus,  to  develop  a polynomial  heuristic  for  the  general  machine  case,  we 
only  need  to  determine  the  W-jobs.  The  completion  of  the  schedule  with  E-jobs  and  T-jobs 
is  straightforward.  A generic  description  of  the  algorithm  is  as  follows: 

Algorithm  4:  Heuristic  for  the  parallel  machine  case 

Step  1.  Load  machines  M1,  M 2 , . . .,  Mrn  with  W-jobs  according  to  a greedy  fashion. 

index  = 1 (index  of  next  job  to  be  considered) 

FOR  i - 1 TO  m DO 

{VW(< th)  «-  0 

Pi  = 0 (time  consumed  by  these  W-jobs) 

WHILE  (Pi  + Pindex  < d - e)  DO 


{ 1 ) * Pi  T Pindex 

W'(oH)^W'(<jH)\JiJtndex} 
index  <—  index  + 1} 


Wl(aH)  <-  Wi(aH)  U {Jindex} 
index  <—  index  + 1} 
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Step  2.  Generate  cumulative  weights  cw . ..,cw™w  in  nondecreasing  order  similiarly  to  Step 
1 in  Algorithm  2 while  each  weight  is  repeated  m times.  Assign  the  remaining  jobs 
(the  non-  W-jobs)  accordingly. 

Step  3.  After  determining  the  W-jobs , the  E-jobs , and  the  T-jobs  for  each  machine,  the  loca- 
tion of  the  common  due  window  and  the  starting  times  for  the  jobs  can  be  determined 
such  that  the  starting  times  are  nonnegative,  Properties  1-3  are  satisfied,  and  that 
there  is  one  machine  which  starts  processing  at  time  0. 


Remark  8 In  Step  2,  there  are  groups  of  m identical  cumulative  weights  since  there  are  m 
machines.  Multiple  solutions  exist  and  the  jobs  can  be  assigned  to  any  machine  as  long  as 
their  position  results  in  the  same  cumulative  weight.  Therefore,  a secondary  criterion  can 
be  used  to  break  these  ties.  See  also  Emmons  [38]. 

Theorem  4 The  absolute  error  for  the  heuristic  schedule  oH  above  is 

m * min (pmax,  d - e)  * min(a,  /3) 


The  relative  error  bound  vanishes  as  the  number  of  jobs  increases.  That  is, 


f.m  cost(crH)  — cost(o*)  n 

imn-4oo  77  rr  ~ 0 

cost(o* ) 


where  cost(oH)  is  the  objective  value  for  schedule  oH  and  costjo*)  is  the  optimal  ob- 
jective value.  It  is  assumed  that  m is  fixed  and  that  cost(o*)  for  the  relative  error  to  be 
defined. 


Proof.  Note  that  the  heuristic  schedule  oH  satisfies  Equation  (3.6)  according  to  Step 
1 in  Algorithm  4 and  Properties  1-3  according  to  Step  3.  Therefore,  we  have  nw  — m < 
n(WI(oH))  < nw  according  to  Lemma  9. 
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Let  a*  be  an  optimal  schedule  which  satisfies  Properties  1-3  and  Equation  (3.6).  There- 
fore, by  Lemma  9,  we  conclude  that  \n(WIH)  - n(WI(o*))\  < m - 1 holds. 

For  each  job  Jj,  let  p f and  p*  be  the  units  of  processing  time  of  job  Jj  which  are 
processed  during  the  due  window  in  schedules  crH  and  <7*,  respectively.  Note  that  we  have 

n n 

X>f  = m * (d  — e)  = ^2p*  (3.12) 

1=1  j=i 

by  definition  of  p f and  p *,  and  by  cost(o*)  > 0.  Using  Lemma  5 and  the  definition  of  pf 
and  p*j,  the  difference  in  the  objective  value  can  be  written  as  follows: 

m 

cost(oH)  — cost(cr*)  = ^]{{  ^2  cw(Jj,oH)  * pj}  — cw(Wl,crH)  * (d  — e)} 

8'=i  J3eJS'(aH) 

m 

~ Y cw(Jj,a*)*pj}  -cw(W\o*)*(d-  e)} 

«=  1 
m 

- !]{{  Y cw(Jj,aH)*{pj  -pf)}} 

*=1 

m 

- Y cw(Jp<j*)*(pj 

»=1  Jj€JS'(a*) 

n 

= Yicw(JP  °H)  * (Pj  - pf)  - cw(  Jj,o*)  * (Pj  - p*)} 
j= 1 

Now  note  that  jobs  with  pf  = p*  = 0 do  not  make  a nonzero  contribution  to  the  cost 
difference  since  a job  Jj  with  pf  = pj  = 0 is  either  an  E-job  or  a T-job  and  has  the  same 
cumulative  weight  in  both  schedules.  Define  the  index-set  IS  by 

IS  = {Jj  : pf  > 0 or  pj  > 0}. 

Accordingly,  we  can  write 

cost(oH)  - cost((T*)  = J2  {cw(Jj,  oH)  * (Pj  - pf)  - cw(Jj,o*)  * (Pj  - pj)}. 

J3eis 

Further  note  that  for  all  jobs  Jj  with  pj  > d - e,  we  have  cw(Jj,oH)  — cw(Jj,o*)  since 
their  cumulative  weight  solely  depends  on  the  number  of  E-jobs  preceeding  or  the  number 
of  T-jobs  succeeding  them.  Therefore,  we  can  write 

cost(crH)  — cost(o*)  = ^2  { cw(Jj,  aH)  * (min(d  - e,pj)  — pf) 

Jj  e is 

- cw(Jj,o*)  * (min(d  - e,pj)  - pj)}.  (3.13) 
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Now  consider  the  cumulative  weights  in  the  heuristic  which  are  applied  to  the  first  m 
non-  Wl-jobs.  We  have  by  Lemma  6 and  by  Lemma  11  that  these  differ  by  at  most  min(a,  j3) 
for  any  two  machines.  The  same  statement  also  holds  for  the  first  m non-  Wl-jobs  in  the 
optimal  schedule.  Due  to  the  fact  that  the  number  of  Wl-jobs  in  schedules  a*  and  differs 
by  at  most  m — 1,  the  cumulative  weights  of  the  first  m non-  Wl-jobs  of  a*  and  of  a11  differ 
by  at  most  min(a,/3). 

After  rearranging  the  terms  in  Equation  (3.13),  we  obtain 
cost(aH)  — cost(a*) 

< min(a,/?)  * (min(d  — e,pj)  — pf) 

JjElS 

Jj€lS 

= min(q,/3)  * {(  min (d  - e,pj))  — m * (d  — e)} 

Jj  e is 

According  to  the  definition  of  IS,  we  have  IS  = lT(cr*)UW((rH).  The  quantity  min(d- 

e,Pj ))  - m * (d  - e)  is  the  total  processing  time  by  which  the  W-jobs  of  either  schedule  a* 
or  (jH  exceed  all  the  due  windows.  This  is  bounded  above  by  the  processing  time  units  of 
the  m largest  W-jobs  and  we  have 

( min(d  - e’Pi))  ~m*{d-e)<m*  min((d  - e),pmax). 

Jjeis 

Therefore,  the  absolute  error  is  bounded  above  by 

m * mm(pmax,  d - e)  * min(a,  /?). 

Now  we  consider  the  relative  error.  We  do  this  only  for  schedules  with  positive  objective 
value.  For  this,  we  produce  a rough  lower  bound  for  any  instance  with  positive  cost  in  an 
optimal  schedule.  Clearly,  for  all  non  W-jobs  except  the  largest  m ones,  the  cumulative 
costs  are  1 * a,  2 * a,  . . .,  1 * /?,  2 * /3,  ....  Therefore,  all  E-jobs  and  T-jobs , except  the  m 
largest  ones,  have  a cumulative  weight  which  is  at  least  min Therefore,  we  have  for 
the  optimal  objective  value: 

n—m, 

cost(cr*)  > min(a,/?)  E pj 
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Using  the  expression  for  the  absolute  error,  we  have 


cost(aH)—cost(o* ) ^ 

cost(cr*) 


< 


m*min(pmax,d- e)*min(ai,/3) 
cost  (a* ) 

m*m'm(prnax  ,d—  e)*min(ct,/3) 
min 


s m*(d—e) 


< 

< 


m*(d— e) 

n— (2*(m+l)+n(WI*)) 

m*(d— e) 

n— (2*(m+l)+m*(d— e)) 


The  last  inequality  is  derived  from  the  assumption  that  jobs  have  integral  processing 
times,  i.e.,  pj  > 1 and  from  the  definition  of  nw.  Therefore,  if  the  number  of  machines  is  a 
given  parameter,  the  relative  error  approaches  zero  as  the  number  of  jobs  increases: 

cost(crH)  — cost(a*) 


limr 


cost(cr*) 


= 0 


QED 


3.6  Summary 

In  this  chapter,  we  have  extended  the  concept  of  due  window  scheduling  to  parallel 
machines.  It  has  been  shown  that  even  the  decision  variable  unit  weights  for  two  machines 
is  NP-hard.  This  justifies  the  use  of  dynamic  programming  and  heuristics.  A dynamic 
programming  algorithm  was  provided  for  the  two  machine  case.  The  non- polynomial  portion 
of  the  algorithm  is  the  determination  of  Wl-jobs.  A simple  heuristic  for  this  part  has 
been  used  to  achieve  polynomial  complexity.  The  heuristic  version  for  parallel  machines  is 
especially  suited  for  multi- criteria  optimization  due  to  multiple  common  cumulative  weights. 
If  the  number  of  jobs  is  reasonably  large,  the  relative  error  in  the  objective  function  tends 
to  be  small  since  the  relative  error  vanishes  as  the  number  of  jobs  increases. 


CHAPTER  4 

OPTIMAL  CONTROL  OF  A PRODUCTION  SYSTEM  WITH  PERIODIC  MAINTENANCE 


4.1  Introduction 

Production  scheduling  is  an  important  area  both  for  researchers  and  for  practitioners, 
which  is  reflected  in  the  vast  amount  of  existing  literature.  For  a survey,  see,  for  example, 
Lawler  et  al.  [75].  A large  section  of  the  existing  literature  approaches  the  problem  with 
the  assumption  that  the  machines  are  reliable  during  the  period  of  consideration.  Machine 
breakdown  and  machine  maintenance  oftentimes  are  not  considered,  mostly  for  simplifica- 
tion. 

Since  the  early  1980s,  there  has  been  an  increased  interest  in  developing  production 
flow  control  models  for  systems  with  unreliable  machines.  Kimemia  and  Gershwin  [66] 
have  studied  a flexible  manufacturing  system  of  unreliable  machines  where  they  used  a 
production  flow  model.  In  the  model  of  Kimemia  and  Gershwin,  the  machine  state  is 
governed  by  a finite  state  Markov  chain,  and  part  movement  is  assumed  to  be  continuous. 
Their  main  contribution  is  the  qualitative  characterization  of  an  optimal  control  structure. 
They  introduced  the  notion  of  hedging  point.  The  hedging  point  is  a production  surplus  level 
which  is  utilized  to  hedge  against  random  machine  failures.  The  control  policy,  also  referred 
to  as  hedging  policy  or  threshold  policy,  operates  as  follows:  There  is  a target  inventory 
level  (referred  to  as  threshold  level  or  hedging  level),  and  the  system  tries  to  reach  this 
level  as  soon  as  possible.  When  the  actual  inventory  is  below  the  target  level,  production 
is  at  full  capacity.  When  the  actual  inventory  is  above  the  target,  the  machine  is  shut 
down.  When  the  target  inventory  level  is  reached,  the  production  rate  equals  the  demand. 
Further,  based  on  assumptions  concerning  the  frequency  of  events  (demand  changes  occur 
less  frequently  than  machine  failures  and  operation  times  are  small  compared  to  the  time 
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between  failures),  they  have  proposed  a multi-level  hierarchical  control  algorithm.  The  top 
level  is  used  for  long-term  decisions  such  as  the  selection  of  the  hedging  point,  the  next  level 
is  used  to  control  the  production  flow  and  the  routing.  At  the  lowest  level  of  control  are 
part-dispatch  policies. 

Subsequently,  the  concept  of  modeling  production  as  continuous  flow,  the  concept  of 
hedging  point,  and  the  concept  of  a hierarchical  control  algorithm  have  been  refined  and 
applied  to  similiar  problem  settings.  Gershwin,  Akella,  and  Choong  [49]  have  improved  the 
algorithms  for  the  various  levels  of  the  hierarchical  control.  Akella,  Choong,  and  Gershwin 

[4]  have  studied  the  performance  of  this  model  for  an  IBM  printed  circuit  card  assembly 
line.  They  have  concluded  that  a hierarchically  structured  policy  such  as  the  one  given 
in  [49]  is  very  effective  in  scheduling  a FMS  and  that  it  can  achieve  high  output  with 
low  WIP  and  can  cope  with  changes  and  disturbances.  Recently,  Bai  and  Gershwin  have 
studied  finite  buffer  capacity  models  in  three  settings.  They  have  considered  one-part-type 
systems  in  [14],  multiple-part-type  systems  in  [15],  and  reentrant  systems  in  [16].  Sharifnia, 
Caramanis,  and  Gershwin  [101]  and  Srinivatsan  and  Gershwin  [103]  have  studied  unreliable 
systems  with  non- negligible  setup  times. 

It  is  difficult  to  obtain  a closed-form  analytical  solution  to  the  optimal  production  flow 
control  problem  for  a system  which  consists  of  unreliable  machines.  In  most  of  the  pre- 
vious research  in  this  area,  only  qualitative  characteristics  and/or  heuristic  solutions  were 
obtained.  In  few  cases,  researchers  have  succeeded  in  deriving  the  solution  to  optimality. 
There  are  two  papers  found  in  the  literature  which  deal  with  similiar  problems  as  the  one  in 
this  study  and  present  closed-form  optimal  solutions.  The  first  one  is  by  Akella  and  Kumar 

[5]  where  the  authors  considered  a manufacturing  system  consisting  of  one  unreliable  ma- 
chine which  produces  one  part-type.  They  analytically  derive  an  exact  solution  and  show 
that  the  optimal  control  policy  is  a hedging  policy. 

The  second  reference  where  a control  problem  has  been  solved  to  optimality  is  by  Hu  and 
Xiang  [60].  In  that  paper,  the  authors  have  considered  a system  consisting  of  one  machine 


56 


where  the  operative  time  is  deterministic  and  the  length  of  the  downtime  is  random.  Their 
work  focuses  on  the  steady  state  for  which  a switching  curve  policy  is  presented.  It  is 
characterized  by  a single  parameter  which  corresponds  to  a target  inventory  level.  It  is 
to  be  reached  at  the  end  of  each  uptime  interval.  The  difference  to  the  hedging  policy  is 
that  the  system  maintains  zero  inventory  level  as  long  as  possible.  Before  the  end  of  each 
uptime  interval,  production  proceeds  at  the  maximum  rate  in  order  to  reach  the  target 
inventory  level  at  the  end  of  the  uptime  interval.  Since  the  length  of  the  uptime  interval  is 
deterministic,  the  inventory  does  not  need  to  be  built  up  in  advance,  whereas  the  hedging 
policy  requires  that  the  system  maintains  the  target  inventory  level  at  all  times,  since  in 
that  case,  the  length  of  the  uptime  interval  is  random. 

In  this  chapter,  we  consider  a single-machine  system  where  maintenance  is  performed 
periodically.  When  the  machine  is  undergoing  maintenance,  the  system  is  shut  down  and 
cannot  produce.  Both  the  uptime  interval  and  the  maintenance  interval  are  deterministic. 
In  contrast,  Akella  and  Kumar  [5]  model  both  the  uptime  and  the  downtime  interval  as 
intervals  of  random  length  and  Hu  and  Xiang  [60]  model  the  downtime  interval  with  random 
length.  In  addition,  the  solutions  in  [5]  and  [60]  are  based  on  the  Bellman  equation,  while 
our  solution  is  obtained  via  Pontryagin’s  Minimum  Principle.  We  will  see  that  the  optimal 
policy  for  our  problem  is  of  the  type  of  switching  curve  policy  rather  than  hedging  policy 
due  to  the  deterministic  length  of  machine  uptime.  In  addition  to  the  steady  state  solutions, 
we  will  also  discuss  the  transient  solution. 

We  define  the  general  N-cycle  problem  in  Section  4.2.  In  Section  4.3,  we  solve  the  one- 
cycle  problem  using  Pontryagin’s  Minimum  Principle.  The  results  for  the  one-cycle  problem 
are  used  to  construct  the  optimal  solution  to  the  two-cycle  problem  in  Section  4.4.  The 
original  N-cycle  problem  is  solved  in  Section  4.5.  The  steady  state  and  the  transient  state 
is  discussed  in  Section  4.6.  Concluding  remarks  are  given  in  Section  4.7. 
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a(t) 


0 a a + 6 2a + 6 2a + 26  T = iV(a  + 6) 


Figure  4.1.  Machine  Status  a(t). 


4.2  Problem  Definition 

We  consider  a system  consisting  of  one  machine  which  produces  one  part-type.  The 
demand  rate  is  constant  and  is  denoted  by  d.  The  machine  requires  periodical  maintenance. 
During  the  maintenance,  the  machine  is  shut  down  and  cannot  produce.  The  time  between 
maintenances  is  constant,  denoted  by  a,  and  the  maintenance  time  is  also  constant,  denoted 
by  b. 

We  denote  the  machine  status  by  a binary  variable  a(t)  which  equals  0 if  maintenance  is 
performed  at  time  t and  1 if  the  machine  is  operative  at  time  t.  Without  loss  of  generality, 
we  assume  a(0)  = 1. 

We  consider  a finite  planning  horizon  from  0 to  T = TV(a  + 6),  where  TV  is  a given  integer 
and  is  greater  or  equal  to  1.  For  TV  cycles,  the  machine  status  is  governed  by 


a(t ) = < 


1, 


if  t G [(j  - l)(a  + 6),  (j  - l)(a  + 6)  + a],  j = 1, . . . , TV, 


(4.1) 


0,  if  te  ((j-1  )(a  + b)  + a,j(a  + b)),  ; = 1,...,TV. 

Figure  4.1  illustrates  the  change  of  machine  status.  We  shall  notice  that  the  time  interval 
of  operation  [0,  a]  is  closed  and  the  maintenance  interval  (a,  a + b)  is  open. 

The  control  variable  is  the  production  rate  u{t).  A production  control  u{t)  is  admissible 


if 


u(t)  G [0,  Ua(t)\,  for  t G [0,  T], 


(4.2) 


where  U is  the  machine  capacity  or  maximum  production  rate. 
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For  the  sake  of  definiteness,  we  restrict  ourselves  to  right-continuous  production  controls, 
i.e.,  production  controls  which  satisfy 


u*(r)  = 


lim  u*(t) 

t-fT,t>T 


for  points  of  discontinuity. 

We  require  that  the  demand  be  feasible.  The  system  is  capable  of  meeting  the  demand 
if  all  > (a  + b)d,  or  equivalently, 


d < 


a + b 


U. 


(4.3) 


We  do  not  consider  the  case  d > -^U.  Such  a case  is  of  little  interest  from  a control 
perspective,  since  the  demand  is  greater  than  the  system  capacity. 

The  production  surplus  x{t)  is  defined  as  the  cumulative  difference  between  total  pro- 
duction and  total  demand.  It  satisfies 


dx(t) 

dt 


— u(t)  - d. 


For  a given  initial  state  *(0)  = xo,  we  have 

x(t)  = x0  + j (u(t)  - d)dt,  for  t € [0,  T],  (4.4) 

Our  objective  is  to  keep  the  cumulative  production  close  to  the  cumulative  demand. 
In  terms  of  the  production  surplus  x(t),  our  objective  is  to  keep  the  production  surplus 
x(t ) close  to  zero.  The  following  differentiable  and  strictly  convex  penalty  function  g(x(t )) 
reflects  this  objective, 

9(x(t))  = x2{t).  (4.5) 


We  integrate  the  penalty  function  over  the  time  interval  [0,  T\  to  obtain  the  performance 
measure.  Both  the  trajectory  and  the  performance  measure  depend  on  the  production 
control.  We  have 


J(u) 


j: 


l(t)dt. 


(4.6) 


We  now  can  formally  define  the  optimal  production  control  problem: 
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Problem  P^:  Find  an  admissible  production  control  u*N(t)  that  causes  the  system 


where  T = N(a  + 6)  is  a given  constant.  The  initial  state  x(0)  is  fixed  and  the  final  state 
x(T)  is  free. 

Here  and  elsewhere  in  this  chapter,  we  use  a subscript  to  indicate  the  number  of  cycles 
involved  in  a problem.  For  example,  (.)  and  w,j(.)  are  the  optimal  cost  and  control  policy 
for  Problem  P\\  J^i-)  and  w^.)  are  the  optimal  cost  and  control  policy  for  Problem  P2 ; etc. 
However,  the  subscript  is  omitted  in  the  derivation  when  there  is  no  possible  ambiguity.  It 
is  used  only  when  comparison  is  made  for  results  of  various  number  of  cycles. 

In  the  following  sections,  we  solve  for  the  optimal  solution  to  the  problem  above.  In 
Section  4.3,  we  consider  the  one-cycle  problem  which  is  obtained  by  letting  N = 1 in  the 
definition  of  Problem  Pjv  (see  Equation  (4.7)).  We  then  use  the  results  from  the  one-cycle 
problem  to  solve  the  two-cycle  problem  in  Section  4.4  and  finally  extend  the  results  to  solve 
the  iV-cycle  problem  for  N > 3 in  Section  4.5. 


In  the  previous  section,  we  formulated  an  optimal  control  problem  for  the  system  under 
consideration.  In  this  section,  we  study  a special  case  of  such  an  optimal  control  problem 
which  has  only  one  up-down  cycle  ( N = 1).  This  one-cycle  problem  will  be  solved  to 
optimality  via  Pontryagin’s  Minimum  Principle.  The  results  will  be  extended  to  the  two- 
cycle  problem  in  the  next  section. 


to  follow  a trajectory  x*(t)  which  minimizes  the  performance  measure 


(4.7) 


4.3  A One-Cvcle  Problem  and  its  Optimal  Solution 
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For  t G (a,  a + b),  since  the  machine  is  undergoing  maintenance,  there  is  no  control 
decision  to  make,  i.e.,  we  have 

u(t)  = 0,  for  t G (a,  a + 6), 

(4.8) 

x(t ) = x(a)  — d(t  — a),  for  t G (a,  a -f  b). 

The  cost  during  the  interval  (a,  a + 6)  can  therefore  be  expressed  in  dependence  of  x(a). 
We  introduce  the  terminal  cost  function  hi(x(a))  by  defining 

/i!(a;(a))  = f (x(a)  - d(t  - a))2dt  = f (x(a)  - dt)2dt.  (4.9) 

J a JO 

Notice  that  h\(x(a))  satisfies 

dx(a)^  1(X(Q))  ~ / 2(a;(a)  - dt)dt  = 2x(ci)b  - db2  = 2(x(a)  - d^)b  - 2x(a  + ^)6.  (4.10) 
The  control  problem  for  the  one-cycle  case  can  be  restated  as  follows. 

Problem  Pj:  Determine  an  admissible  control  «J(<),  i.e.,  «J(i)  G [0,  U],  for  t G [0 ,a],  that 
causes  the  system 


dx(t) 

dt 


u(t)  — d 


to  follow  a trajectory  x*{t)  which  minimizes  the  performance  measure 

Ji(u)  ~ [ x2{t)dt  + hi(x(a)).  (4.11) 

Jo 

The  boundary  conditions  are  a:(0)  = x0  and  x(a)  free. 

For  Problem  Pi,  the  Hamiltonian  (see  Pontryagin  [86],  p.  18  and  Kirk  [67],  p.  188)  is 
given  by 

H(x(t),u(t),p(t))  = x2{t)  +p(t)[u(t)  - d],  for  t G [0,a], 

where  the  function  p(t)  is  referred  to  as  costate  function. 

According  to  Pontryagin’s  Minimum  Principle,  necessary  conditions  for  an  optimal  con- 
trol law  u*(t)  (see  Pontryagin  [86],  p.  69  and  Kirk  [67],  p.  233)  are 
dx*(t ) 


dt 

dp*(t) 


dH 
dp 
dU 


(x*(t),u*(t),p*(t)y, 


( Xm(t),u*(t),p*(t )); 


dt  dx 

H(x*(t),u*(t),p'(t))  < H{x*(t),u(t),p*(t)), 


for  all  admissible  u(t); 
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and  the  boundary  condition  (see  Kirk  [67],  p.  200) 

~{x\T))-p\T)  = 0. 
These  evaluate  in  case  of  Problem  P\  to 


dx*(t) 

dt 

= u*(t)  — d ; 

(4.12) 

dp*(t) 

dt 

= ~2 

(4.13) 

*\t)  + P*(t)[u*(t)  - d] 

= +p*(t)[u(t)  - d}} 

u(t)e[o,U] 

= x*2(t)-p*(t)d+  min  {p*(t)u{t)}; 

u(t)e[o,u] 

(4.14) 

2 x*(a)b  - db 2 - p*(a) 

= 0. 

(4.15) 

Since  the  final  time  is  fixed  and  the  Hamiltonian  does  not  explicitly  depend  on  time,  the 
Hamiltonian  must  satisfy  the  following  condition  for  the  optimal  control  u*{t),  the  optimal 
trajectory  and  the  costate  function  p*(t), 

7i(x*(t),  u*(t),p*(t))  = ci,  for  all  f, 

where  ci  is  a constant  (see  Pontryagin  [86],  p.  19  and  Kirk  [67],  p.  236). 

For  Problem  Pi,  we  have 

x*2(t)  + p*(t)[u*(t)~  d\  = ci,  for  t G [0,  a].  (4.16) 


Since  there  is  only  one  term  in  the  Hamiltonian  involving  u(t),  we  can  evaluate  u(t) 
according  to  Equation  (4.14)  of  Pontryagin ’s  Minimum  Principle.  This  results  in 


0,  ifp*(f)>0, 

'(0=4  u,  ifP*(t)<o, 

(3,  if  p*(t)  = 0, 


(4.17) 


where  f3  6 [0,  U]  is  to  be  determined. 

According  to  Equation  (4.17),  the  optimal  control  law  u*(t)  is  thus  determined  whenever 
we  have  p*(t)  ^ 0.  In  the  case  that  roots  exist  for  the  function  p*{t)  within  the  interval  [0,  a], 
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we  further  distinguish  between  singular  intervals , switching  times , and  switching  points. 
These  terms  are  defined  in  the  following. 

An  interval  [fi,  t2]  with  t\  < t2  is  called  a singular  interval  if  the  following  two  conditions 
are  satisfied: 


(i)  p*(t)  = 0 for  te[ti,t2\,  and 

(ii)  p*(t ) = 0 for  with  t[  < t2  < t'2  (4-18) 

if  and  only  if  t\  = t\  and  t'2  = t2. 

In  case  of  t\  > 0,  the  definition  implies  that  there  must  exist  a neighborhood  to  the  left 
of  h in  which  p*(t)  is  nonzero,  i.e.,  there  exists  an  e > 0 such  thatp*(f)  ^ 0 for  t € (ti~e,ti). 
Similiarly,  in  case  of  t2  < a,  the  definition  implies  that  there  must  exist  a neighborhood  to 
the  right  of  t2  in  which  p*(t)  is  nonzero,  i.e.,  there  exists  an  0 0 such  that  p*(t)  ^ 0 for 
t € (t2,  t2  + c). 

A time  to  is  called  a switching  time  if 


(i)  P*(f  0)  = 0,  and 

(ii)  to  is  not  part  of  any  singular  interval. 


(4.19) 


That  is,  t0  is  an  isolated  time  instant  at  which  p*{t0)  = 0.  We  do  not  consider  0 and  a 
as  switching  times,  i.e.,  to  must  satisfy  to  € (0,a).  The  production  surplus  x(t0)  is  referred 
to  as  a switching  point. 

In  the  following,  we  will  consider  the  implications  for  a control  which  has  a singular 
interval  and  subsequently,  we  study  the  implications  for  a control  which  has  a switching 
point.  We  will  show  that  an  optimal  control  will  either  have  one  singular  interval  and  no 
switching  point,  or  one  switching  point  and  no  singular  interval,  or  neither  of  them. 


4.3.1  Singular  Intervals 


Now  assume  that  there  is  a singular  interval  [fl5i2]  C [0,a],  i.e., 


p*(t)  = 0, 


for  t £ [*i,/2]- 


(4.20) 
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Using  Equations  (4.13)  and  (4.12),  we  conclude  that  the  optimal  trajectory  and  the  optimal 
control  policy  must  satisfy 


x*(t)  = 0,  for  t G (*i,t2),  (4.21) 

u*{t)-d  = 0,  for  t G (h,t2).  (4.22) 

Thus,  for  a singular  interval,  we  must  have  (3  - d in  Equation  (4.17).  From  this  equation, 
we  conclude  that  the  optimal  control  u*(t)  is  a piecewise  constant  function  for  t G [0,a]. 
From  Equation  (4.4),  we  have  that  the  optimal  trajectory  x*(t)  is  a continuous  and  piecewise 
linear  function  for  t G [0,a].  This  implies  x*(ti)  - x*{t2)  = 0,  so  that  Equation  (4.21)  can 
be  extended  from  (t\,t2)  to  [t\,t2]\ 

x*(t)  = 0 for  t G [ti,t2\.  (4.23) 

Using  our  convention  that  we  only  consider  right- continuous  production  rates,  we  can 
extend  Equation  (4.22)  from  (t\,t2)  to  [ti,t2): 

u*(t)  = d,  for  t G [h,t2). 

We  now  proceed  to  analyze  the  implications  of  having  a singular  interval  and  consider 
the  constant  c\  in  Equation  (4.16).  According  to  Equations  (4.20)  and  (4.23),  we  have 
ci  = 0,  and  thus, 

x*2(t)  + p*2(t)(u*(t)  - d)  = 0,  for  t G [0,  a}. 

Clearly,  a root  of  the  costate  function  p*(t)  requires  a zero  production  surplus  level.  In 
other  words,  if  the  production  surplus  level  is  nonzero  at  a time  t ( x*(t ) ^ 0),  the  time  t 
cannot  be  a root  of  the  costate  function  p*(t),  i.e.,  p*(t)  ^ 0,  and  therefore,  t cannot  be 
contained  in  a singular  interval. 

Therefore,  if  there  exists  a singular  interval,  then  during  this  singular  interval  and  any 
other  singular  interval,  and  also  at  any  switching  point,  we  must  have  zero  production 
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(a)  Optimal  trajectory  in  a neighborhood  of  [<i,t2]- 


(b)  Two  singular  intervals. 


Figure  4.2.  A Control  with  Singular  Interval, 
surplus  level  in  order  to  satisfy  Equation  (4.16).  That  is, 


if  there  is  a singular  interval  C [0,a], 

then  p*(f)  = 0 implies  x*(t)  = 0,  for  t € [0,  a].  (4.24) 

In  the  following,  we  demonstrate  that  there  is  at  most  one  singular  interval,  and  further, 
we  demonstrate  that  if  there  is  a singular  interval,  then  there  is  no  switching  point. 

Consider  that  control  u*(t)  has  a singular  interval  [fj , <2],  and,  for  simplicity,  assume 
further  t\  > 0 and  < a.  The  remaining  cases,  in  which  t\  = 0 or  t2  — a,  can  be  analyzed  in 
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the  same  way.  Figure  4.2(a)  illustrates  the  possible  trajectories  for  a singular  interval  [#i , *2] 
with  0 < < t2  < a.  According  to  Condition  (4.24),  we  have  x*(t)  = 0 for  t £ [fi,i2]* 

According  to  the  definition  of  a singular  interval,  we  have  p*(t)  ^ 0 in  a neighborhood 
outside  of  the  singular  interval  [<i,f2],  i.e.,  there  exists  an  e > 0 such  that  p*(t)  7^  0 for 
t £ (t\  — e, fi)  U (t2,t2  + e).  The  optimal  control  law  according  to  Equation  (4.17)  thus 
requires  u*(t)  ^ d for  t £ (fi  — e,  tj)  U (t2,t2  + e).  In  particular,  we  have  u*(t)  = U for 
p*(t)  < 0,  and  u*(t)  = 0 for  p*(t)  > 0.  These  production  levels  are  represented  in  Figure 
4.2(a)  by  segments  (1)  to  (4).  Segment  (1)  represents  the  case  u*(t)  = 0 for  t £ ( t\  — e,  fi), 
segment  (2)  represents  the  case  u*(t)  = U for  / £ ( t\  - e,/i),  segment  (3)  represents  the 
case  u*(t ) — U for  t £ (t2,t2  + e),  and  finally,  segment  (4)  represents  the  case  u*(t)  = 0 for 
t £ (t2,t2  + e). 

We  now  show  that  there  is  at  most  one  singular  interval  within  [0,a].  To  the  contrary, 
assume  that  there  is  another  singular  interval  [ta,tb]  within  [0,  a}.  By  Condition  (4.24), 
during  the  singular  interval  [$„,<&],  the  production  surplus  level  must  be  zero  as  well.  A 
possible  scenario  is  shown  in  Figure  4.2(b).  From  Figure  4.2(b),  we  see  further  that  there 
must  be  a switching  point  with  nonzero  production  surplus  level  between  the  two  singular 
intervals.  This  contradicts  Condition  (4.24).  Therefore,  an  optimal  trajectory  can  have  at 
most  one  singular  interval. 

We  now  show  that  it  is  not  possible  that  an  optimal  control  has  a singular  interval  and 
a switching  time  to  £ (0,a).  Assume  to  the  contrary  that  there  is  a switching  point  at  time 
t0  G (0,o)  in  addition  to  the  singular  interval  [t\,t2\.  By  Condition  (4.24),  we  must  have 
x(to)  = 0.  In  a similiar  way  as  above,  we  conclude  that  there  must  also  be  another  switching 
point  with  nonzero  production  surplus  level  between  the  singular  interval  and  the  switching 
time  to-  This  again  violates  Condition  (4.24).  We  conclude  that  an  optimal  control  cannot 
have  both  a singular  interval  and  a switching  point. 

Note  that  for  t\  = 0 or  t2  = a,  we  can  apply  similiar  reasoning  to  arrive  at  the  same 
conclusion.  Thus,  if  an  optimal  control  has  a singular  interval  [t\,t2]  C [0,a],  it  is  necessary 


66 


that  [ti,t2]  is  the  only  singular  interval  and  that  no  switching  point  exists.  In  summary, 
we  have  observed  that  until  the  trajectory  reaches  the  zero  production  surplus  level,  the 
optimal  control  level  is  either  U (if  the  initial  production  surplus  is  negative)  or  0 (if  the 
initial  production  surplus  is  positive).  Subsequently,  the  singular  interval  starts.  When 
the  singular  interval  ends,  the  control  will  either  be  U or  0.  Thus,  we  conclude  that  it  is 
necessary  that  the  optimal  control  satisfies 


u*(t) 


0\,  ifte[0,fi), 
d,  ifie[ti,t2), 

02,  if  te[t2,a), 


(4.25) 


where  0\  and  02  may  take  the  value  of  0 or  U . 

We  now  show  that  if  the  optimal  control  has  a singular  interval,  another  necessary 
condition  for  optimality  is  02  = U. 

Before  we  proceed,  we  first  establish  a property  which  we  will  use  to  compare  the 
performance  of  two  policies,  u'(t)  and  u''(t). 


If  two  admissible  controls  u'(t)  and  u"(t)  lead  to  trajectories  x'(t)  and  x"(t) 
such  that  (a)  |ar,(t)|  < |x"(/)|,  for  t £ [0,u],  and 
(b)  either  0 > x\a)  > x"(a ), 
or  db  < x\a)  < x"(a), 

then,  we  have  Ji(u')  < J\(u").  (4.26) 


To  see  the  validity  of  this  property,  notice  that  we  have 


J\(u')  = f (x'(t))2dt  + f (x'(a)  — dt)2dt  < f (x"(t))2dt  + [ (x'(a)  - dt)2dt, 

Jo  Jo  Jo  Jo 

since  |ar'(t)|  < |*"(/)|,  for  t £ [0,a].  Further,  both  0 > x'(a)  > x"(a)  and  db  < x'(a)  < x"(a) 

imply  ( x'(a ) — dt)2  < ( x”(a ) — dt)2  for  t £ [0, 6] , and  therefore,  we  have  J\(u')  < Ji(u"). 

We  now  use  Property  (4.26)  to  demonstrate  @2  = U in  Equation  (4.25)  by  showing  that 

02  = 0 does  not  lead  to  an  optimal  control.  Note  that  we  only  need  to  consider  t2  < a, 
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since  the  interval  ( t2,a ) vanishes  for  t2  = o.  In  that  case,  (d2  is  of  no  interest.  For  t2  < a, 
Equation  (4.25)  with  /32  = 0 leads  to  a control  up2=0(t)  with 


up2=o  (t)  = < 


Pi, 

d, 

0, 


if  t g [o,ti), 
if  t € [h,t2), 
if  t G [t2,a). 


and  a trajectory  xp2=0(t ) with 


(Pi  - d)(t-ti),  if  i G [0,  <i], 


xth=o(t ) = < 

We  now  modify  the  production 
policy  is 


0,  if  t G [<i,<2]> 

—d(t  — t2),  if  t G [^2?  o]- 

control  up2=o(t)  by  letting  (52  = d.  The  resulting  control 


Up  3=d(t)  = < 


Pi, 
d, 
d , 


if  t G [0,ti), 
if  t G [<1,^2), 
if  t G [*2,a), 


and  leads  to  a trajectory  xp2=d(t)  with 


xp2=d(t)  = < 


(Pi  ~ d)(t-t  1), 

0, 

0, 


if  t G [0,ti], 
if  t G [^,<2]) 
if  t G [t2,a\. 


Note  that  we  have  \xp2=d(t)\  < \xp2=0(t)\  for  t G [0,a],  and  0 = xp2=d(a)  > xp2=0(a).  By 
Property  (4.26),  we  therefore  have  J\(up2=d)  < J\(up2= 0).  Thus,  up2=0(t)  is  not  optimal, 
and  consequently,  (32  = U in  Equation  (4.25)  is  a necessary  condition  for  optimality. 
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In  summary,  we  conclude  that  if  u*(t)  is  a right-continuous  optimal  control  which  has  a 
singular  interval  [/i , ^2]  1=  [0,a],  it  is  necessary  that  u*(t)  satisfies 


u*(t)  = < 


Pi, 

d, 

U, 


if  t G [0,  /1), 
if  t € [^ii  ^2)5 
if  t G [t2,a), 


(4.27) 


where  /3\  G {0,  U}. 

Please  note  that  an  optimal  production  control  which  has  a singular  interval  is  com- 
pletely specified  by  the  parameters  i1?  t2,  and  fi\. 


4.3.2  Switching  Times  and  Switching  Points 


We  now  consider  the  case  that  there  exists  a switching  time  to  G (0,a).  According  to 
our  observations  above,  we  know  that  an  optimal  control  cannot  contain  a singular  interval 
and  a switching  point  at  the  same  time.  In  the  following,  we  further  show  that  there  can 
be  at  most  one  switching  point.  In  addition,  we  show  that  the  switching  point  must  satisfy 
x*(to)  > 0 and  that  the  control  must  switch  from  0 to  U . 

Recall  that  a switching  time  to  satisfies  p*(f0)  = 0 and  represents  the  transition  of  p*(t) 
from  a negative  to  a positive  level  or  vice  versa.  According  to  Equation  (4.17),  we  only 
need  to  consider  a switch  of  control  from  U to  0 or  from  0 to  U . We  further  distinguish 
positivity  and  negativity  of  the  production  surplus  level  x(to)  at  the  switching  time  to  and 
show  that  switches  from  U to  0 and  switches  with  x(t0)  < 0 are  never  optimal.  Table  4.1 
presents  the  cases  for  which  we  will  show  nonoptimality. 

Switching  from  U to  0 at  t0  G (0,a)  where  x(t0)  > 0: 

In  the  following,  we  show  that  a control  with  a switching  point  satisfying  x(t0)  > 0, 
switching  from  U to  0 is  not  optimal  by  providing  an  improved  control  u'(t).  The  idea 
is  to  truncate  the  trajectory  around  the  switching  point  so  that  the  modified  trajectory 
stays  closer  to  the  zero  production  surplus  level  (see  Figure  4.3(a)).  In  this  case,  there  is  a 
neighborhood  around  t0  where  the  production  control  is  U before  t0  and  0 after  t0.  Further, 
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Figure  4.3.  Switching  Points  which  are  not  optimal. 
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the  production  surplus  at  t0  is  positive.  Specifically,  there  exists  an  e G (0,  such  that 
x(t ) > 0 for  t £ [to  — to  + e]  and  u(t)  satisfies 


«(/)  = < 


U, 


0, 


if  t € [to  to), 

if  t E [to,  to  + e). 


Notice  that  we  have  x(f0  — = x{to  + e).  Regardless  of  the  production  control  u(t) 

for  t ^ [t0  — jTZrfC,  to  + e),  we  show  that  u(t ) is  not  optimal  by  improving  the  control  during 
the  interval  [to  — jjz^e,  to  + e).  The  improved  control  u'{t)  is  obtained  as  follows: 


u\t) 


u(t),  if  t £ [t0  - ^e,  t0  + e), 

d,  if  t G [to  — j7Zrff»to  + e). 


Consequently,  we  have 


x'(t) 


x(t), 

x(t0  + e), 


if  t ^ [^0  [/_^e>to  + c], 

if  t G [to  — fjz^Cto  + f]- 


This  implies  that  we  have 


di(tt,)=  / {x\t))2dt  + /ii(x'(a))  < / (x(t))2dt  + /ii(x'(a)), 

Jo  Jo 

since  x,2(t)  < x2(f)  for  t G [0,a]  and  ar'2(t)  < x2(f)  for  t G (t0  - jtz^Mo  + e).  Since 
*'(a)  = ®(a),  we  have  = fii(x(a))  and  thus  Ji(u')  < J\{u).  Therefore,  a control 

which  switches  from  U to  0 at  a switching  point  with  x(f0)  > 0 cannot  be  optimal. 


Table  4.1.  Classification  of  Switching  Points. 


Switching 

x(t0)  < 0 

x(t0)  = 0 

x(to)  > 0 

from  U to  0 

not  optimal 

not  optimal 

not  optimal 

from  0 to  U 

not  optimal 

possibly 

optimal 

possibly 

optimal 
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Switching  from  0 to  U at  t0  G (0,a)  where  x(t0)  < 0: 


It  can  be  shown  that  a control  which  switches  from  0 to  U at  a switching  point  satisfying 
x(to)  < 0 is  not  optimal  either.  The  idea  is  to  provide  an  improved  control  u'(t)  so  that 


the  modified  trajectory  x'(t ) is  closer  to  the  zero  production  level.  This  can  be  done  in  a 
similiar  way  as  above  (see  the  dashed  line  in  Figure  4.3(b)). 

Switching  from  U to  0 at  t0  G (0,a)  where  x(t0)  - 0: 

We  show  in  two  steps  that  a switch  from  U to  0 at  time  fQ  with  switching  point  x(to)  = 0 
is  not  optimal.  First,  we  show  that  there  is  no  other  switching  time  except  t0.  Second,  we 
show  that  the  production  control  which  has  no  singular  interval  and  which  has  exactly  one 
switch  at  to  with  x(to)  = 0 is  not  optimal  by  providing  an  improved  control. 

Now  assume  that  there  is  another  switching  time  in  addition  to  the  switching  time  f0 
where  the  control  switches  from  U to  0.  Figure  4.3(c)  illustrates  this  situation.  The  dashed 
line  segments  illustrate  the  situation  that  there  is  a switch  at  time  t'  (before  Z0)  and  that 
there  is  a switch  at  time  t"  (after  to).  Thus,  if  we  have  a second  switching  point  either 
before  to  or  after  to,  Figure  4.3(c)  demonstrates  that  a second  switching  point  would  be  a 
switching  point  which  switches  from  0 to  U for  negative  production  surplus.  But  we  have 
just  concluded  above  that  a switch  from  0 to  U at  negative  production  surplus  level  is  not 
optimal.  We  therefore  conclude  that  there  can  be  no  switching  time  other  than  to-  A control 
u(t)  which  switches  from  U to  0 at  x(to)  = 0 has  exactly  one  switching  point, 


Next,  we  show  that  such  a control  is  not  optimal  by  providing  an  improved  control  u'(t). 
Consider  the  control  u'{t)  with 


u\t)  = -I 


u{t),  iff  e [0,fo), 

d,  if  f G [f0,  a), 


which  leads  to  a trajectory  x'[t)  with 
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x\t ) = 1 


*(<)>  if  t e [o,  i0], 

0,  if  t £ [*0,a]. 

Thus,  we  have  |s'(*)|  < |*(i)|  for  t £ [0,a],  and  0 = x\a)  > -d(a-t0)  = x(a).  According  to 
Property  (4.26),  we  have  Ji(u')  < J\(u).  In  conclusion,  a switching  point  at  zero  production 
surplus  level  which  switches  from  U to  0 is  not  optimal. 

Switching  from  U to  0 at  t0  £ (0,a)  where  x(t0)  < 0: 


In  the  same  way  in  which  we  showed  above  that  a switch  from  U to  0 at  a zero  production 
surplus  level  is  not  optimal,  we  can  also  show  that  a switch  from  U to  0 at  a negative 
production  surplus  level  is  not  optimal. 

Summary 


Thus  far,  we  have  shown  that  four  of  the  six  cases  in  Table  4.1  are  not  optimal.  Therefore, 
an  optimal  control  cannot  have  a switching  point  with  negative  production  surplus,  nor  can 
a control  which  switches  from  U to  0 be  optimal.  Therefore,  if  an  optimal  control  has  a 
switching  point,  it  is  necessary  that  the  control  switches  from  0 to  U and  that  the  production 
surplus  at  the  switching  time  is  nonnegative.  We  conclude  that  an  optimal  control  can  have 
at  most  one  switching  point,  since  a second  switching  point  would  have  to  switch  from  U to 
0.  Further,  nonnegative  production  surplus  level  at  the  switching  time  (x*(t0)  > 0)  requires 
positive  initial  production  surplus  level  (a;(0)  > 0),  since 


z(0)  = x*(t0)  + dt0  > dt0  > 0,  for  t0  £ (0,  a). 

Thus,  if  u*(t)  is  an  optimal  control  with  a switching  time  tg  € (0,  a),  necessary  conditions 
for  the  switching  point  and  the  initial  production  surplus  level  are 

z*(*o)  > 0,  (4.28) 


and 


a:(0)  > 0 


(4.29) 
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Table  4.2.  Initial  production  surplusses  and  initial  control  levels 


II 

o 

'is* 

s 

o 

II 

a. 

£ 

o 

II 

o 

®(0)  < 0 

not  optimal 
(Case  Al) 

not  optimal 
(Case  A2) 

r(0)  = 0 

not  optimal 
(Case  A3) 

not  optimal 
(Case  A4) 

z(0)  > 0 

not  optimal 
(Case  A5) 

not  optimal 
(Case  A6) 

and  it  is  further  necessary  that  u*(/)  satisfies 


u*(t)  = - 

4.3.3  Initial  Control  Level  n*(0) 


0, 

U, 


if  t e [o,t0), 

if  t e [t0,  a). 


(4.30) 


We  now  examine  the  initial  control  level  u*(0)  in  dependence  of  the  initial  production 
surplus  level  r(0).  We  only  need  to  consider  the  three  control  levels  d,  0,  and  U,  since 
a singular  interval  requires  u*(t)  = d,  and  if  p*(t)  ± 0,  we  have  u*(t)  = 0 or  u*(t)  = U 
according  to  Equation  (4.17).  In  the  following,  we  will  show  u*(0)  = 0 for  z(0)  > 0, 
u*(0)  = d for  x(0)  = 0,  and  w*(0)  = U for  x(0)  < 0 by  showing  that  all  other  cases  are  not 
optimal.  The  non-optimal  cases  are  numbered  in  Table  4.2. 

Case  Al: 

In  this  case,  we  have  u( 0)  = d for  an  initial  production  surplus  z(0)  < 0.  Since  a control 
level  of  d can  only  occur  within  a singular  interval  and  since  we  have  established  that 
a singular  interval  can  only  occur  at  zero  production  surplus  level  (see  Property  (4.24)), 
u(0)  = d cannot  be  optimal  for  r(0)  < 0. 
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Case  A2: 

In  this  case,  we  have  u(0)  = 0 for  an  initial  production  surplus  x(0)  < 0.  We  first  note 
that  such  a control  cannot  have  a switching  point,  since  the  initial  production  surplus  is 
negative  (compare  Equation  (4.29)).  Thus,  the  production  control  u(t)  satisfies 

u(t)  — 0,  for  t £ [0,  a). 

We  define  an  alternative  control  u_(f)  by 

u~(t)  = d,  for  t £ [0,  a).  (4.31) 

The  controls  lead  to  trajectories  x(t ) and  x_(t),  respectively,  which  satisfy 


x(t)  = x(0)  - dt  < x(0)  < 0,  for  t £ [0,a], 


and 

x-(t ) = x(0)  < 0,  for  t £ [0,  a]. 

We  conclude  |ar(0)|  = |x_(f)|  < \x(t)\  for  t £ [0,  a],  and  0 > x_(a)  = x(0)  > x(0)-da  = x(a). 
By  Property  (4.26),  we  have  Ji(u_)  < J\(u)  and  thus,  u(0)  = 0 is  not  optimal  for  x(0)  < 0. 
Case  A3: 

In  this  case,  we  have  u(0)  = U for  an  initial  production  surplus  x(0)  = 0.  From  the 
discussion  in  the  previous  section,  such  a control  cannot  have  a switching  point,  since  an 
optimal  switch  never  switches  from  U to  0.  Thus,  the  production  control  u(t)  satisfies 

u(t)  = U,  for  t £ [0,  a). 


It  leads  to  a trajectory  with 
x(t)  = < 


( U-d)t , 

( U — d)a  — d(t  — a), 


for  t £ [0,  a], 
for  t £ [a,  a + 6]. 


We  now  show  that  this  trajectory  is  not  optimal  and  distinguish  two  cases,  d < -^U  (Case 
A3-1)  and  d = U (Case  A3-2). 
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Case  A3-1: 


In  case  of  d < -^^U,  we  define  an  alternative  control  u+(t)  by 


«+(*)  = < 


d,  if  t e [0,a- 


U,  lit  £[cl  - j^,a). 


It  leads  to  a trajectory  with 


db  — d(t  — a), 


0, 


for  f G [0,a-  ^], 

for  t G [a- 

for  t G [a,  a + 6]. 


Note  that  we  have  x+(t)  > ®+(0)  = 0 for  f G [0,a]  and  x+(a)  = db.  Since  u+(t)  < u(t ) 
for  t € [0,  a],  we  have  0 < x+(t)  < x(t)  (and  thus  \x+(t)\  < |z(t)|)  for  t G [0,a],  as  well  as 
x(a)  = x(0)+(U-d)a  > db  = x+(a).  According  to  Property  (4.26),  we  have  J\{u+)  < J\(u). 

Case  A3-2: 

In  case  of  d = -^U,  we  define  a class  of  controls  t(t)  by 


Notice  that  x(t)  is  identical  to  xAt(t)  for  At  = 0.  Since  we  have  x(a)  = (U  — d)a  = db 
and  xAt(a)  — db  — (U  — d)At  < db  for  At  > 0,  we  cannot  apply  Property  (4.26).  We  directly 
use  the  definition  of  the  performance  measure  and  obtain 


uAt(t)  = < 


d,  if  t G [0,  At), 


U,  if  t G [At,  a), 


where  At  > 0 is  to  be  determined.  It  leads  to  a trajectory  with 


0, 


for  t G [0,  At], 


**«(<)  =<  (U  - d)(t  - At), 


for  t G [At,  a], 


(U  - d)(a  - At)  - d(t  - a),  for  tG[a,a  + b ]. 
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We  take  the  derivative  with  respect  to  At  and  obtain 


d rb 

— Ji(nAj)  = {-1){(U  - d)(a- At)}2  + -2(U -d){{U -d)(a- At)- dt}dt 

= ~(U  - d)2(a  - At)2  - 2 (U  - d)2(a  - A t)b  + (U  - d)db 2 

= —(U  — d)2 {{a  — At)2  + 2(a  — At)6  — — ^ — -62} 

= -({/-d)2{(a-At  + 6)2--^62}.  (4.32) 

From  d = ^^77 , we  obtain  6 = ^j^a.  Replacing  b in  Equation  (4.32)  results  in 


U 


dAt 


Ji(uAt)  = -(£7  - d)2{(A_a  _ At) 


V U-d  2 

^ — «>  ) 


U - d 1 d 


U 2 U(U  - d) 


= ~(U -d)2{^a2 -2^aAt  + (At)2} 

= -(77  - d)2{-^a(a  - 2At)  + (At)2} 
a 

< —(77  — d)2{^a(a  — 2A<)}. 


)a2  - 2~aAt  + (At)2} 


That  implies  -^J\(uAt)  < 0 for  At  < |.  Since  u(t)  is  identical  to  uAt(t ) for  At  = 0, 
we  conclude  that  the  policy  u(t ) is  not  optimal.  This  concludes  the  considerations  for  Case 


A3-2. 


Case  A4: 

In  this  case,  we  have  u(0)  = 0 for  an  initial  production  surplus  a:(0)  = 0.  In  a similiar 
way  as  in  Case  A2,  we  can  show  that  u_(t)  (Equation  (4.31))  leads  to  a trajectory  with 
improved  performance,  and  thus,  u(0)  = 0 for  x(0)  = 0 is  not  optimal. 

Case  A5: 

In  this  case,  we  have  u(0)  = U for  an  initial  production  surplus  x(0)  > 0.  For  the  same 
reason  as  in  Case  A3,  we  conclude  that  the  optimal  control  u(t)  satisfies 


u{t)  = U, 


for  t € [0,  a), 


which  leads  to  a trajectory  with 
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x(t)  = < 


x(0)  + (U-d)t,  for  t G [0,  a], 

x(0)  + (U  - d)a  - d(t  - a),  for  t G [a,  a + b]. 

We  distinguish  the  two  cases  *(0)  > db  (Case  A5-1)  and  x(0)  € (0 ,db)  (Case  A5-2). 

Case  A5-1: 

For  x(0)  > db , we  define  an  alternative  control  Ud(t)  by 

Ud{t)  = d,  for  t G [0,a). 


It  leads  to  a trajectory  with 

*(0), 

x(0)  - d(t  — a), 
We  have  |a:d(f)|  < |x(t)|  for  t G [0,a]  and 


xd(t)  = i 


for  t G [0,  a], 
for  t G [a,  a + b\. 


db  < Xd(a)  = x(0)  < z(0)  + (17  — d)a  = x(a). 

Therefore,  J\(ud)  < J\(u)  according  to  Property  (4.26). 

Case  A5-2: 

In  this  case,  we  have  a:(0)  G (0,<f6).  We  define  an  alternative  control  u+(t)  by 

d,  if<€[0,o-^^), 

U,  ift€[a-^2gl,o). 

Notice  that  u+(t)  is  well-defined,  since 

db  — a:(0)  _ a(U  — d)  - db  + z(0) 


“+(0  = < 


a — 


>0, 


U -d  U -d 

and  a — < a.  The  production  rate  u+(t)  leads  to  a trajectory  with 


*+(<)  = < 


x(0), 

x(0)  + (U-d)(t-(a-d-^l)), 
db  — d(t  — a), 


for  t G [0,n-  ^j^], 
fortG[o-^?,fl], 
for  t G [a,  a + 6]. 
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Note  that  we  have  z+(t)  > x+(0)  > 0 for  t e [0,a]  and  z+(a)  = db.  Since  u+(t)  < u(t ) 
for  t G [0,  a],  we  have  0 < x+(t)  < x(t)  (and  thus  |z+(f)|  < |*(t)|)  for  t 6 [0,a],  as  well  as 
x(a ) = z(0 )+(U-d)a  > db  = x+(a).  According  to  Property  (4.26),  we  have  Ji(u+)  < J\(u). 

Case  A6: 

In  this  case,  we  have  u(0)  = d for  an  initial  production  surplus  x(0)  > 0.  Since  a control 
level  of  d can  only  occur  within  a singular  interval  and  since  we  have  established  that  a 
singular  interval  can  only  occur  at  zero  production  surplus  level,  u(0)  = d cannot  be  optimal 
for  x(0)  > 0. 

In  summary,  we  have  for  the  initial  control  level  u*(0): 


0,  if  x(0)  > 0, 

w*(°)  = d,  if  s(0)  = 0,  (4-33) 

U,  if  a;(0)  < 0. 

Thus,  for  nonzero  initial  production  surplus,  the  system  initially  moves  toward  zero 
production  surplus  level.  If  the  production  surplus  is  initially  zero,  we  see  that  the  optimal 
control  will  have  a singular  interval  starting  at  0. 


4.3.4  Optimal  Solution 


In  the  previous  subsections,  we  have  seen  that  there  are  three  possible  structures  for  an 
optimal  control.  The  optimal  trajectory  either  has  a singular  interval  or  a switching  point 
or  neither  of  them.  We  have  also  seen  that  the  initial  control  level  u*(0)  is  determined 
by  the  initial  production  surplus  x(0)  (Equation  (4.33)).  In  this  section,  we  solve  for  the 
optimal  solution  to  the  one-cycle  problem  P\  with  respect  to  the  initial  production  surplus 
level  a:(0).  We  first  start  with  the  case  x(0)  = 0.  The  optimal  control  for  this  case  will 
provide  a basis  to  proceed  with  the  cases  z(0)  < 0 and  x(0)  > 0.  The  range  of  the  initial 
production  surplus  is  partitioned  as  follows: 


Bl: 


B2: 


x(0)  = 0, 

*(0)  € (—(17  — d)(a  — Cr),  0), 
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B3:  a:(0)  < -(U  - d)(a  - Cr), 

B3-1:  *(0)  G [~{U  - d)a , -( U - d)(a  - Cr)\, 

B3-2:  x(0)  < -(U  - d)a , 

B4:  x(0)  > d(a  - Cr), 

B4-1:  x(0)>rf(G+^), 

B4-2:  x(0)  G [d(a  - Cr),  d(a  + ^)), 

B5:  x(0)  e (0,  d(a  - Cr)), 

where  Cr  = K\J TT^d  ~ derived  in  Section  4.3.4. 

In  each  case,  we  examine  which  of  the  three  control  structures  is  optimal.  In  order  to 
conveniently  refer  to  the  structures,  we  number  them  as  follows: 


1.  Type-1  Structure:  The  optimal  trajectory  has  a singular  interval  [<! , /2] - According 
to  Equation  (4.27),  we  have  for  the  optimal  control 


where  (3\  6 {0,  U}. 


u*(t)  - i 


Pu 

d, 

U, 


if  t e [o,ti), 

if  t e [ h,t2 ), 

if  t e [t2,a), 


2.  Type-2  Structure:  The  optimal  trajectory  has  a switching  point  at  time  t0.  Ac- 
cording to  Equation  (4.30),  we  have  for  the  optimal  control 


u*(t) 


Please  note  that  we  have  in  particular 


if  t e [0,  t0), 
if  t £ [t0,  a). 


u*(0)  = 0.  (4.34) 

3.  Type-3  Structure:  The  optimal  trajectory  has  neither  a singular  interval  nor  a 
switching  point.  Since  the  costate  function  p*{t)  is  continuous,  we  thus  have  not  only 


p*{t)  ^ 0,  but  also 
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either  p*{t ) < 0, 
or  p*(t)  > 0, 

This  implies  for  the  optimal  control 

either  u*(t)  = 0, 

or  u*(t)  = U , 
Please  note  that  this  requires 


for  t £ [0,  a], 
for  t £ [0,a]. 


for  t £ [0,  a], 
for  t £ [0,  a]. 


«*(0)  £ {0,U}. 


Case  Bl:  adOi  = 0 


(4.35) 


(4.36) 


According  to  Equation  (4.33),  we  have  u*(0)  = d.  Due  to  Properties  (4.34)  and  (4.36), 
this  excludes  a Type-2  Structure  and  a Type-3  Structure  from  consideration.  Consequently, 
the  optimal  trajectory  has  a Type-1  Structure  which  has  a singular  interval  [fi,t2].  Since 
«*(0)  = d,  the  singular  interval  has  the  left  endpoint  tx  = 0.  Substituting  t\  = 0 in  Equation 
(4.27),  we  obtain 

d,  if  t £ [0,  f2)> 

«(0  = { (4.37) 


I U,  if  t £ a), 

where  € ( 0,  ct].  In  order  to  simplify  the  derivation,  we  introduce  parameter  C by  defining 
C = a — t2.  The  parameter  C therefore  satisfies  C £ [0,a).  Substituting  t2  by  a - C in 
Equation  (4.37)  results  in 


d, 

U, 


if  t £ [0,a  - C ), 
if  t £ [a  - C,  a), 


which  leads  to  the  trajectory  (see  Figure  4.4), 


(4.38) 


x(t)  = < 


0, 

(U-d)(t-(a-C)), 
(U  - d)C  - d(t  - a). 


if  t £ [0,  a - C ), 
if  t £ [a  - C,  a), 
if  t £ [a,  a + b). 
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We  now  seek  to  find  the  control  with  the  best  performance  measure.  Since  both  the 
optimal  control  and  the  trajectory  are  characterized  by  the  parameter  C,  the  performance 
measure  can  also  be  expressed  in  terms  of  C.  We  write  J\{C)  in  place  of  Ji(u), 

Ji(C)=  f x2{t)dt  + h1(x(a))  = [C({U -d)t)2dt+  [b((U  - d)C  - dtfdt.  (4.39) 
-'0  Jo  Jo 

We  seek  to  determine  the  value  of  C £ [0,  a)  which  solves 


min{Ji(C)  : C £ [0,a)}. 


(4.40) 


Therefore,  we  differentiate  J\{C)  by  using  Leipniz’  rule 

i /(!)’  f(X’ t)M  = f(X’  i{x)]^x)  ~ l(*Mx))±-xa(z)  + ±t(x, t)dt , 

and  obtain 


^Ji(C)  = ((U  - d)C)2  + jb  2(U  - d){(U  - d)C  - dt)dt 
= (U  - d)2{C2  + 2bC  - jj~~jb2}. 

The  function  is  of  a quadratic  form  with  roots 

C',  = -h-  Jb*  + (T^2, 

c‘;=-b+fiu£p, 

where  the  smaller  root  is  negative. 


(4.41) 


(4.42) 


d2 
dC 2 


Ji(C)  = 2(U -d)\C  + b), 


Since 
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we  have 


&MCr)  < o, 
w*MC?)  > o. 


(4.43) 


Since  we  minimize,  the  unconstrained  minimum  is  obtained  for  C — C".  We  rename  C" 
as  Cr  and  have 

and 

-^Ji(Cr)  = (U-  d)\2Cr  + 26)  = (U  - d)\2b^-^—d)  > 0.  (4.44) 

It  is  not  difficult  to  see  that  CT  > 0 since  > 1.  Further,  using  Condition  (4.3) 

(d  < -^U),  we  also  see  that 


Thus,  Cr  satisfies 


K^fZd  i)  < K^u^u-i)  - 

y/by/a  + 6-6  < (y/a  + 6)2  - 6 = a. 


Cr  = b(\ 


U 


U -d 


1)  G (0,  a). 


(4.45) 


Consequently,  CT  solves  the  problem  (4.40).  This  means  that  the  singular  interval  is 
[0,  a — Cr\  and  the  optimal  control  law  satisfies 


u*(t)  — < 


d, 


U, 


if  t £ [0,a  - Cr), 

if  t G [a  - Cr,  a). 


(4.46) 


Since  we  will  frequently  refer  to  the  optimal  trajectory  for  x(0)  = 0,  we  name  it  x^t). 
During  the  maintenance  interval  (a,  a + 6),  we  have  u*(t)  = 0 and  together  with  Equation 
(4.46),  we  obtain  for  the  optimal  trajectory  Xq(<), 


Xq(0  = < 


0, 

(U-d)(t-(a-Cr)), 

( U - d)Cr  - d(t  - a), 


if  t £ [0,  a - Cr\, 
if  t e [a  - Cr,a], 
if  t e [a,  a + 6], 


(4.47) 
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For  the  costate  function  p*(t),  we  have  p*(t ) = 0 for  t £ [0,a  - CT\.  Using  Equation 
(4.13),  we  have  for  t £ [a  — Cr,a\, 


P*{t)  = p*(t)  - p*{a  - Cr) 

= —2  / x(t)cIt 

Ja-Cr 

= -2  (U  - d)(r  - (a  - CV))dr 

Ja-Cr 

rt-(a-Cr) 

= —2  / (U  — d)rdr 

Jo 

= -(U  -d){t-(a-Cr))2. 


In  summary,  we  have 


P*(0  = < 


0, 


if  t € [0,  a - Cr], 
-(U  — d)(t  — (a  — CT))2,  if  t e [a  - Cr,a], 
We  verify  the  boundary  condition  (see  Equation  (4.15)): 


2{z*(a)}&-d&2-{p*(a)}  = 2{(U  - d)Cr}b  - db2  - {-(U  - d)C2} 

d 


(U  - d){C2  + 2bCr - 

0 


(4.48) 


U -d 

(see  Equation  (4.41)).  (4.49) 


The  optimal  control  u*(t ),  the  optimal  trajectory  x*(t ),  and  the  costate  function  p*(t) 
are  given  in  Figure  4.5. 

We  will  see  that  Cr  is  not  only  of  significance  in  case  of  :c(0)  = 0.  We  will  show 
that  for  all  x(0)  € ( —{U  — d)(a  — Cr),d(a  — Cr)),  the  optimal  control  will  result  in  a 
trajectory  with  a singular  interval  [U,<2]  where  the  right  endpoint  of  the  singular  interval 
is  t2  = a — Cr-  A control  which  only  assumes  the  minimum  or  the  maximum  production 
rate  is  called  a bang-bang  control.  We  will  see  that  for  an  initial  production  surplus  level 
with  x(0)  ^ {-(U  - d)(a  - Cr),d(a  - CT)),  we  will  have  a bang-bang  control  (with  at  most 
one  switching  point)  as  optimal  policy.  A bang-bang  control  with  Type-2  Structure  has  one 
switching  point  and  a bang-bang  control  with  Type-3  Structure  does  not  have  a switching 
point. 
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Figure  4.5.  Optimal  Solution  for  Case  Bl:  a;(0)  = 0. 
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This  concludes  the  analysis  for  the  case  where  £(0)  = 0.  We  now  proceed  with  an 
optimality  condition. 

Optimality  Condition 

At  this  point,  we  will  state  an  optimality  condition  for  Problem  P\  which  will  be  used 
in  the  reasoning  of  the  remaining  cases. 

Optimality  Condition 

If  x\(t)  and  x\ (f)  are  two  optimal  trajectories  for  Problem  P\  with  initial  production  sur- 
passes »i(0)  and  £2(0)5  respectively,  then  the  trajectories  will  not  cross  each  other,  i.e., 

if  x\{t')  < £2 (0  for  some  t'  E [0,a],  then  xl(t)  < £?j(t)  for  all  t E [0,  a],  (4.50) 

and  in  particular  for  t'  = 0,  we  have 


if  £i(0)  < £2(0),  then  x\ (t)  < x^t),  for  t E [0,a]. 


(4.51) 


Explanation:  We  first  note  that  the  performance  measure  (see  Equation  (4.6))  is  strictly 


convex,  since 


[ x2(t)dt  = f ■ ^ x2(t)dt  = / 2 dt  — 2T  > 0. 

Jo  Jo  dxl(t)  Jo 


dx2(t)  Jo  Jo  dx2(t)‘ 

This  implies  that  the  optimal  trajectory  is  unique  for  each  initial  production  surplus.  If  the 
trajectories  never  meet,  they  don’t  cross  each  other  either.  And  if  the  trajectories  meet  at 
a time  t',  they  will  coincide  from  then  on  since  the  optimal  trajectory  is  unique. 

Case  B2:  £ ( 0 1 E 1-iU  - d)(a  - CAP) 

First,  since  £(0)  < 0,  the  initial  control  level  satisfies  w*(0)  = U according  to  Equation 

(4.33) .  Next,  we  show  that  the  optimal  control  has  a Type-1  Structure: 

A Type-2  Structure  can  be  excluded,  since  it  requires  w*(0)  = 0 according  to  Property 

(4.34) . 

A Type-3  Structure  according  to  Equation  (4.35)  would  imply 
u(t ) = U,  forte[0,  a], 

x(a  — Cr)  = £(0)  + {U  — d)(a  — CT)  > 0,  for  £(0)  E (—(17  - d)(a  - Cr),  0). 


86 


Such  a trajectory  would  cross  trajectory  Xq (t)  (see  Equation  (4.46)),  since 

*(0)  < 0 = Xq(0), 

x(a  — Cr)  > 0 = Xq(o,  — Cr). 

Therefore,  we  can  also  exclude  a Type-3  Structure  from  consideration. 

In  conclusion,  we  must  have  a Type-1  Structure,  i.e.,  the  optimal  trajectory  has  a 
singular  interval.  We  now  determine  the  left  and  the  right  endpoint  of  the  singular  interval 
[^l , ^2] - The  left  endpoint  t\  is  the  time  when  the  production  surplus  reaches  the  zero  level, 
i.e.,  t\  = 9^.  At  this  point,  the  trajectory  for  x(0)  G (-( U - d)(a  - CT),  0)  meets  with 
Trajectory  xj(t).  By  the  Optimality  Condition  (Equation  (4.51)),  the  optimal  trajectory 
for  x(0)  G (— (U  — d)(a  — Cr),0)  during  the  interval  , a]  is  identical  to  Trajectory  Xq (t). 
This  means  that  the  right  endpoint  of  the  singular  interval  is  £2  = a — CT. 

Therefore,  we  have  for  the  optimal  control  u*(t), 


o,  if  te  [0,^0). 


(4.52) 


U,  if  t G [a  - Cr,a). 
The  optimal  trajectory  x*(t)  satisfies 


if  t e [0, 


0 


(4.53) 


(U  — d)(t  — (a  — CT )),  if  t G [a  - Cr,a] 


( U - d)CT  — d(t  - a),  if  t E [a,  a + 6]. 


From  the  definition  of  a singular  interval,  we  have  p*(t)  = 0,  for  t G [ , « - Cr\. 


From  Equation  (4.13),  we  obtain  for  t G [0, 
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Since  the  trajectory  x*(t)  for  t £ [-y_) , a + 6]  is  identical  to  the  trajectory  in  Equa- 
tion (4.47),  the  costate  function  p*(t)  in  the  remaining  interval  yields  the  same 

expression  for  p*(t)  as  in  Equation  (4.48).  In  summary,  we  have 


p*(t)  = 0, 

-{U  -d)(t-{a-Cr))\ 


if  t € [0, 

iOeRrStn-CJ. 

if  t 6 [a  — Cr,  a]. 


(4.54) 


Since  the  expressions  for  x*(a)  and  p*(a)  are  the  same  as  in  Case  Bl,  the  boundary 
condition  (see  Equation  (4.15))  is  verified  in  the  same  way  as  in  Equation  (4.49).  The 
optimal  control  u*(t),  the  optimal  trajectory  x*(t),  and  the  costate  function  p*(t)  are  shown 
in  Figure  4.6. 


Case  B3:  g(0)  < -(U  - d)(a  - C,) 

First,  since  a;(0)  < 0,  the  initial  control  level  satisfies  u*(0)  = U according  to  Equation 
(4.33).  Next,  we  examine  the  control  structure. 

According  to  Property  (4.34),  we  can  exclude  a Type-2  Structure  (a  production  control 
with  a switching  point),  since  it  requires  u*(0)  = 0.  Therefore,  the  optimal  control  has  either 
a Type-1  Structure  or  a Type-3  Structure.  That  is,  the  optimal  control  either  produces 
at  maximum  capacity  or  it  has  a singular  interval.  By  distinguishing  between  the  cases 
a(0)  G [-(17  - d)a,  -(U  - d)(a  - Cr)]  (Case  B3-1)  and  *(0)  < -(V  - d)a  (Case  B3-2),  we 
will  show  that  the  optimal  control  has  a Type-3  Structure. 

Case  B3-1:  x(0)  G [-(17  - d)a,  -(17  - d)(a  - Cr)\. 

In  case  of  a Type-3  Structure,  we  produce  at  maximum  capacity  during  [0,a]: 


u(t)  = 17,  for  t € [0,  a). 


(4.55) 


In  case  of  a Type-1  Structure,  we  have  a singular  interval  [ti,t2].  The  production  control 
satisfies  Equation  (4.27)  where  = "“*(0)  — U.  The  value  of  (3i  is  determined  according 
to  the  initial  production  control  level  (see  Equation  (4.33)).  For  the  left  endpoint  of  the 
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Figure  4.6.  Optimal  Solution  for  Case  B 2:  £(())  6 (— (tf  - d)(a  - Cr),0). 
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x(t) 
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\ 1 
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*(0) 

Figure  4.7.  A possible  trajectory  for  z(0)  G [—(U  - d)a,  —(U  - d)(a  - Cr) ]. 

singular  interval,  we  have  tx  - Let  t2  = a - C where  C € [0,a  - 7^).  With 

[ti,t2]  = [ , a — C],  the  production  control  thus  satisfies 

V , ift€[0,^), 

”(')=!  <J,  ifie[^gl,a-C),  (4-56) 

U,  if  t G [a  — C,  a). 

Notice  that  we  can  represent  the  production  control  in  Equation  (4.55)  by  letting  C = 
a — zr^d~  in  Equation  (4.56).  Thus,  if  we  extend  the  range  of  C from  [0,o  - -jjjQ)  to 
[0) a fj~d  ],  we  have  represented  both  control  structures  which  need  to  be  considered. 

A possible  trajectory  x(t ) is  shown  in  Figure  4.7.  The  performance  measure  can  be 
expressed  as  a function  of  the  parameter  (7, 

Ji{C)  = [ U~d  {(U  - d)t)2dt+  [ {(U-d)t)2dt  + [ ((U  - d)C  - dtfdt.  (4.57) 

Jo  Jo  Jo 


We  seek  to  determine  the  value  of  C which  solves 


min{ JX(C):  CG  [0,a- j^}}-  (4.58) 

Notice  that  the  cost  function  J\(C)  in  Equation  (4.57)  differs  from  the  cost  function 

~JC°) 

in  Equation  (4.39)  only  by  the  term  f0u~d  ((U  - d)t)2dt.  This  term  is  independent  of  C. 
Therefore,  the  derivatives  of  the  cost  functions  in  Equations  (4.57)  and  (4.39)  are  identical 
(see  Equation  (4.41)): 

Ajl(C)  = (f/-d)2{C2  + 26C-zr^62}.  (4.59) 

Since  the  function  is  a quadratic  function  with  negative  function  values  between 

its  two  roots  C'r  and  Cr  = C"  (see  Equation  (4.42)),  i.e., 


4-mc)  < 0, 


for  C G (C;,C")- 


(4.60) 
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Since  the  smaller  root  is  negative  {C'r  < 0),  and  since 
— x(  0^ 

a - < Cr  = C",  for  x(0)  < -{U  - d)(a  - Cr), 

we  have 

(4.61) 

From  Equations  (4.60)  and  (4.61),  we  conclude 

-^MC)  < 0,  for  C E [0,«  - (4-62) 

That  is,  if  C increases  within  the  interval  [0,a-  the  function  J\(C)  decreases. 

Consequently,  C = a - solves  (4.58),  and  Equation  (4.56)  with  C = a — leads  to 

u*(t)  — U,  forte[0,a).  (4.63) 

Case  B3-2:  x(0)  < — ( U — d)a. 

Recall  that  we  have  already  excluded  a Type-2  Structure,  since  it  requires  w*(0)  = 0 for 
the  initial  control  level.  In  this  case,  the  optimal  control  cannot  have  a singular  interval 
either  (Type-1  Structure),  since  any  possible  trajectory  would  satisfy 


x(t ) < z(0)  + (U  - d)a  < 0,  for  t G [0,  a]  and  z(0)  < —(U  - d)a. 


Thus,  we  have  a Type-3  Structure.  Since  the  initial  control  level  satisfies  u*(0)  = U,  we 
have  the  same  optimal  control  as  in  Case  B3-1  (see  Equation  (4.63))  and  produce  at  full 
capacity. 


We  now  consider  the  optimal  trajectory  and  the  costate  function.  Since  we  have  the 
same  optimal  production  control  in  both  cases  B3-1  and  B3-2,  we  do  not  need  to  consider 
these  two  cases  separately. 

The  optimal  trajectory  for  ar(0)  < — (U  — d)(a  — Cr ) satisfies 


x*(t) 


x(0)  + (U  - d)t, 

z(0)  + (U  — d)a  — d(t  — a), 


if  t e [0,  a], 
if  t € [a,  a + 6]. 


(4.64) 
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From  Equation  (4.13),  we  have  p*(a)  - p*(t ) = /“  -2 x*(t)cIt,  or  equivalently, 

P *(t)  = 2 £ X*(T )dr  + p*(a), 

= 2 J\x(0)  + (U  — d)r  )dr  + p*(a), 

= 2x(0)(a-t)  + (U  -d)(a2  -t2)+p*(a),  forte[0,a].  (4.65) 

From  the  boundary  condition  (Equation  (4.15)),  we  obtain 

p*(a)  = 2x*(a)b  - db 2 = 2{z(0)  + (U  - d)a}b  - db 2.  (4.66) 

From  Equations  (4.65)  and  (4.66),  we  obtain 

p*(t)  = 2 (a  + b - t)x( 0)  + {U  - d)(a2  + 2 ab  - t 2)  - d62.  (4.67) 

Notice  that  the  costate  function  is  a monotone  increasing  function  since  p*(t2)  — p*(ti)  = 
//j2  —2 x*(r)dr  > 0 for  ti  < t%  and  that  an  upper  bound  for  the  costate  function  value  is 
—db2,  since 

p*{a)  = 2{a:(0)  + (U  — d)a}b  — db2  < -db2,  for  z(0)  < — ( U — d)a. 

These  observations  for  the  costate  function  p*(t)  are  reflected  in  Figure  4.8.  It  also 
illustrates  the  optimal  control  u*(t)  and  the  optimal  trajectory  x*(t).  This  completes  the 
analysis  of  Case  B3-2  (a:(0)  < —(U  — d)a). 

In  summary,  with  Cases  Bl,  B2,  and  B3,  we  have  now  analyzed  all  cases  which  satisfy 
*(0)  < 0.  We  have  seen  that  the  optimal  control  is  to  produce  at  maximum  capacity  for 
*(0)  < —(U  — d)(a  — Cr),  and  that  there  exists  a singular  interval  for  x(0)  € (~(U  — d)(a  — 
Cr),  0].  In  the  following,  we  will  analyze  Cases  B4  and  B5  which  represent  the  two  cases 
for  which  we  have  x(0)  > 0. 

Case  B4:  adOl  > d(a  - CA 

First,  since  a;(0)  > 0,  the  initial  control  level  satisfies  u*( 0)  = 0 according  to  Equation 
(4.33).  Next,  we  can  eliminate  a Type-1  Structure  from  consideration.  The  reasoning  is  as 


follows: 
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For  any  production  control,  we  have 

x{t)  > x(0)  - dt  > 0,  for  t € [0,  a - Cr)  and  x(0)  > d(a  - Cr ).  (4.68) 

Since  an  optimal  trajectory  x*{t)  for  x(0)  > d(a  - Cr ) does  not  cross  the  Trajectory  Xq (t) 
(see  Equation  (4.51)),  and  since  the  Trajectory  x*0 (t)  satisfies  Xq(1)  > 0 for  t e (a  - Cr,a], 
we  have 

x*(t)  > Xq (t)  > 0,  for  t G (a  - CT,  a]  and  x(0)  > d(a  — Cr).  (4.69) 

From  Equations  (4.68)  and  (4.69),  there  cannot  be  a singular  interval.  We  therefore  exclude 
a Type-1  Structure  from  consideration. 

Thus,  we  have  either  a Type-2  Structure  or  a Type-3  Structure:  Since  we  have  u*(0)  = 0 
for  x(0)  > 0 according  to  Equation  (4.33),  we  either  shut  the  machine  down  during  the  entire 
interval  or  we  switch  the  production  rate  from  0 to  U at  some  time  t0  € (0,a). 

If  we  shut  the  machine  down,  the  control  policy  is 


u(t)  — 0,  for  t € [0,  a). 


(4.70) 


If  we  have  a policy  with  a switching  point  at  time  to  6 (0,  a),  we  introduce  a parameter 
S with  S = a - t0  G (0,a)  to  simplify  the  calculation.  According  to  Equation  (4.30)  with 
switching  time  to  = a — S , we  have 


u(t)  = 


0, 

U, 


if  t G [0,  a — S ), 
if  t e [a  — S,  a). 


(4.71) 


We  now  combine  the  solution  structures.  Note  that  if  we  let  S = 0 in  Equation  (4.71), 
the  interval  [a  — S',  a]  vanishes  and  we  obtain  the  same  policy  as  in  Equation  (4.70). 

Thus,  the  optimal  control  for  x(0)  > d(a  - Cr ) is  specified  by  the  optimal  value  of  S 
where  S £ [0,a).  The  trajectory  is 


x(t)  = < 


x(0)  - dt, 


if  t 6 [0,  a - S ), 


x(0)  — d(a  — S)  + (U  — d)(t  - (a  - S )),  if  t £ [a  — S,  a). 


(4.72) 
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It  is  shown  in  Figure  4.9.  The  performance  measure  Ji(S)  satisfies 

i-S  rS 


Ji(S)  = / (x(0)  — dt)2dt  + / (x(0)  — d(a  — S)  + (U  — d)i)2dt 

Jo  Jo 

rb 

/ (®(0 )-da  + US  -dt)2dt.  (4.73) 

Jo 


+ 


Thus,  we  are  seeking  the  value  of  parameter  S which  solves 


minlJi^)  : S € [0,a)}. 


(4.74) 


For  the  derivative,  we  have  according  to  Leipniz’  rule 


= — l(as(0)  - d(a  - S ))2  + (*(0)  - d(a  - S)  + (U  - d)S )2 

+ [ 2d(x(0)  - d{a  - S)  + (U  - d)t)dt  + [b  2U(x(0)~  da+US  - dt)dt. 

Jo  Jo 


Evaluating  the  integrals  and  rearranging  the  terms,  we  have 

= U(V  + <J){S2  + 2*W-to+UI>s  _ JL-{d{a  + ‘ ) - x(0))}.  (4.75) 

The  two  roots  S[  and  S"  are  given  by 


S[ 


x(0)-da  + Ub  lrx(0)  — da  + Ub  2b 

+ TT+d 


U + d 


U + d 


- \ICyK,>  rr  T7  "U)2  + jrri(d(a  + \)  - x(°))> 


u _ x(0)  — da  + Ub  Lx(0)-da  + Ub 


S?  = 


26 


U + d 


+ '/( — Uj~d — )2  + uTd[d{a  + 2}  - *(0))’ 


and  satisfy 
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The  smaller  root  S[  is  negative  and  satisfies  < 0.  The  larger  root  is  S"  and 

,2 

satisfies  > 0.  Since  our  objective  is  to  minimize,  we  focus  on  S'{.  We  rename  S'" 


as  5i(x(0)): 


_^_(d(a+^)_x(0)).  (4.77) 


According  to  Equation  (4.76),  we  have 


(4.78) 


At  this  point,  we  observe  in  Equation  (4.75)  that  is  negative  between  its  roots, 


In  the  following,  we  consider  the  two  cases  Si(x(0))  < 0 (Case  B4-1)  and  5i(x(0))  > 0 
(Case  B4-2).  We  will  see  that  the  optimal  control  for  Case  B4-1  has  a Type-3  Structure 
while  the  optimal  control  for  Case  B4-2  has  a Type-2  Structure. 

Notice  the  following  relationship  between  x(0)  and  Si(x(0)).  Since  we  have  + 

§)  — x(0))  < 0,  if  and  only  if  x(0)  > d(a  + |),  we  conclude  from  Equation  (4.77) 


Thus,  Case  B4-1  is  represented  by  x(0)  > d(a  + |). 

Further,  Equation  (4.80)  and  5i(a:(0))  > 0 imply  x(0)  < d(a  + |).  Since  Case  B4 
considers  an  initial  surplus  z(0)  satisfying  z(0)  > d(a  - Cr ),  Case  B4-2  therefore  covers 


Z§MS)<  0,  for  5 €(51,510. 


and  positive  for  S > 5i(a:(0))  = 5",  i.e., 


(4.79) 


(4.80) 


x(0  )e[d(a-Cr),d(a+^)). 
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Case  B4-1:  x(0)  > d(a  + |). 

In  this  case,  we  will  show  that  5 = 0 solves  (4.74).  We  first  consider  the  derivative  of 
the  performance  measure  for  5^0.  Since  a;(0)  > d(a  + |)  implies  5i(x(0))  < 0 and 
5i(a;(0))  < 0 and  5 € (0,  a)  imply  5 > 5i(z(0)),  we  have  according  to  Equations  (4.79)  and 
(4.80) 

— </i(5)  > 0,  for  5 £ (0,a)  and  x(0)  > d(a  + -). 
do  2 

That  is,  as  5 increases,  so  does  Ji(5).  Consequently,  5 = 0 solves  (4.74).  Substituting 
5 = 0 in  Equation  (4.71)  leads  to  the  control 

u*(t)  = 0,  fort£[0,a),  (4-81) 

i.e.,  there  is  neither  a switching  point  nor  a singular  interval.  Figure  4.11  illustrates  this 
situation.  The  optimal  trajectory  x*(t ) therefore  satisfies 

x*(t)  = x(0)  - dt,  for  t £ [0,  a + 6].  (4.82) 

From  the  boundary  condition  (see  Equation  (4.15)),  we  obtain  p*(a ) = 2x*{a)b  - db 2. 
Together  with  Equation  (4.13),  this  leads  to 

p*(t)  = 2 ^ x*(t )dr  + p*(a) 

= 2^  (x(0)  - dr)dr  + 2(x(0)  - da)b  - db2 

= 2x(0 )(a  - t)  - d(a 2 - t 2)  + 2(z(0)  - da)b  - db 2 

= 2x(0)(a  + b - t)  — d((a  + b)2  — t2),  fort£[0,a].  (4.83) 

This  completes  our  analysis  for  an  initial  production  surplus  satisfying  a:(0)  > d(a  + |) 

which  corresponds  to  5i(z(0))  < 0.  The  functions  u*(t),  x*(t),  and  p*(t)  are  shown  in 

Figure  4.10. 

Case  B4-2:  2;(0)  £ [d(a  - Cr),  d(a  + |)). 

Recall  that  the  range  for  parameter  5 in  (4.74)  is  [0,a).  In  Appendix  C.l  (see  Equation 
(C.l)),  we  show 

for  a;(0)  £ [d(a  - Cr),  d(a  + ^)). 


5i(a;(0))  < Cr  < a, 
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u*(t) 

U — 

d — 

t 

H 1 — - 

a a + b 


Figure  4.10.  Optimal  Solution  for  Case  B4-l:fl x(O)  > d(a  + |). 
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Since  we  already  have  ^(^(O))  > 0,  this  results  in 


Si(*(0))e(0,Cr]C[0,a),  for  x(0)  £ [d(a  — Cr),  d(a  + ^)).  (4.84) 

Since  the  second  derivative  of  J\(S)  for  S = 5i(x(0))  is  negative  according  to  Equation 
(4.78),  we  conclude  that  5i(x(0))  solves  (4.74)  and  that  the  optimal  control  has  a Type-2 
Structure  with  a switching  point  at  time  t0  - a - S'i(a;(0)). 

However,  in  order  for  the  optimal  control  to  have  a switching  point,  the  production 
surplus  at  the  switching  time  must  be  nonnegative.  In  Appendix  C.2,  we  verify  that  the 
optimality  condition  (see  Equation  (4.28))  is  satisfied, 


x*(a  - Si(z(0)))  > 0,  for  x(0)  € [d(a  - Cr),  d(a  + ^)). 


The  optimal  policy  is  thus 


«*(*)  = 


0,  if  t e [0,  a - ^(^(O))), 


(4.85) 


U,  if  t £ [a  - 5i(z(0)),a). 


For  the  trajectory  x*(t ),  we  have 


x*(t ) = < 


x(0)-dt,  if  / G [0,a-  5i(a;(0))], 

®(0)  — dt  + U(t  — (a  - Si(a;(0)))),  if  t € [a  — 5i(x(0)),  a], 


(4.86) 


:c(0)  - dt  + U ^(^(O)), 


if  t G [a,  a -f  6]. 
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For  the  function  p*(t),  we  have  a root  at  the  switching  point,  i.e.,  p*(a  - 5*i(a;(0)))  = 0. 
Together  with  Equation  (4.13),  we  derive 


P*(t ) = < 


St  SlW0))2a :*(r)dr,  if  t e [0,a  - 51(a:(0))], 

fa-s1(x( o))  -2x*(r)dr,  if  t 6 [a  - 5i(x(0)),a], 


For  t € [0,  a — Si(x(0))],  this  yields 

fa~Si  (^(o)) 


p 


ra-a  ipcpjq  /■ 

'(i)  = jf  2x*(t  )dr  — j 


&—S\  (®(o)) 


2(x(0)  — dr)dr 


2x(0 )(a  - Sx(x(0))  - t)  - d((a  - S^O)))2  - t2). 


For  t G [a  — 5i(x(0)),  a],  this  yields 


P 


’( t ) = f —2  x*(r)dr 

J a—Si(x(0)) 

•t 


2 [ {*(0)  - d(a  - 5!(*(0)))  + (U-  d)(r  - ( a - S1(x(0)))}di 

Ja—S i (x(0)) 


= -2  / {*(0)-  tf(a-Sa(x(0)))  + (t/-d)r}d7 

«/ a— 5i  (a;(0)) 

= -2(a?(0)  — U(a  — Si(a;(0))))(*  - (a  - Si(x(0)))) 

-(V  - d)(t2  - (a  - S^O)))2). 


In  summary,  we  have 

2*(0)(a  - S^O))  - t)  - d((a  - S^O)))2  - t2), 

if  * € [O.a-S^O))]; 

P*W  = — 2(x(0)  - U (a  - 51(*(0))))(t  - (a  - 51(i(0))))  (4-87) 

—(U  — d){t2  — (a  — «!>i(*(0)))2), 

if  t G [a  — 5’i(x(0)),a]. 

Finally,  we  verify  the  boundary  condition  (4.15).  With  x*(a)  = x(0)  - da  + U Si(x( 0)) 


and 


P*(a)  = -2(x(0)-U(a-S1(x(0))))(a-(a-S1(x(0)))) 


—(U  — d){a2  — (a  — S1(x(0)))2) 
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= -2(*(0)  - U(a  - 51(*(0))))51(*(0))  - (U  - d)(2aSi(x(0))  - Si(z(0))2) 

= -( U + d)(S'1(*( 0)))2  - 2(a?(0)  - da)5x(x(0)), 

we  have 

2x*(a)b  - db 2 - p*(a) 

= 2{a:(0)  - da  + US^xi 0))}6  - db 2 + {U  + d)(5i(x(0)))2  + 2(i(0)  - da)Si(x( 0)) 

= (C/  + d)(5!(x(0)))2  + 2(x(0)  - da  + t/6)5’1(a:(0))  + 2(x(0)  - d(a  + b-))b 
= (U  + rf){(5,(x( 0)))2  + 2I(0)~^+  WJ,(X(0))  + 2p^(*(0) - <i(„  + |))(4.88) 
= 0.  (4.89) 

Equation  (4.89)  is  established  by  observing  that  5i(x(0))  is  a root  of  (see 

Equation  (4.75))  and  that  ^Ji(5i(a;(0)))  is  a multiple  of  the  term  on  the  right-hand-side 
of  Equation  (4.88). 

The  functions  u*(t),  x*(t),  and  p*(t)  are  shown  in  Figure  4.12.  This  completes  the 
analysis  for  Case  B4-2  (x(0)  G [d(a-Cr),d(a  + §)))  which  corresponds  to  Si(x(0))  > 0 and 
x(0)  > d(a  — Cr). 

Case  B5:  .t(01  G fO.dta  - C-)) 

First,  since  x(0)  > 0,  the  initial  control  level  satisfies  u*(0)  = 0 according  to  Equation 
(4.33).  We  will  see  that  the  optimal  control  structure  is  a Type-1  Structure.  We  eliminate 
a Type-2  Structure  and  a Type-3  Structure  according  to  the  following  reasoning: 

According  to  the  optimality  condition  (see  Equation  (4.51)),  the  optimal  trajectory  x*(t) 
for  a:(0)  G (0 ,d(a—  Cr))  cannot  cross  the  Trajectory  a:g(i),  i*e., 

x*(t)  > Xq (t)  > 0,  forfG[0,a].  (4.90) 

With  u*( 0)  = 0,  a Type-3  Structure  would  require  u(t)  — 0 for  t G [0,  a)  and  lead  to 

x(a  - Cr)  = z(0)  - d(a  - Cr)  < 0,  for  *(0)  G (0,  d(a  - Cr)). 


Therefore,  such  a control  cannot  be  optimal. 


0 a - Cr  a a + b 


Figure  4.12.  Optimal  Solution  for  Case  B4-2:  x(0)  6 [d(a  - Cr),d(a+  |)). 
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In  order  to  show  that  a Type-2  Structure  is  not  optimal,  we  consider  the  optimal  tra- 
jectory for  x(0)  = d(a  - Cr ) and  in  particular,  we  are  interested  in  the  production  surplus 
level  at  time  t = a-Cr.  According  to  Appendix  C.l  (see  Equation  (C.6)),  we  have 

S^dia-Cr))  = Cr. 

Thus,  we  conclude  that  the  switching  time  for  x(0)  = d(a  - Cr)  is 

a - Si(d(a  - Cr))  = a - Cr. 

This  implies  that  the  optimal  production  surplus  level  for  z(0)  = d(a-Cr ) at  time  t = a-CT 
is  zero: 

x*(a  - CT)  = [d(a  - Cr)}  - d{a  - {Si(d(a  - Cr))})  = 0.  (4.91) 

We  can  now  apply  the  optimality  condition  in  Equation  (4.51)  and  conclude  that  the  pro- 
duction surplus  level  for  an  optimal  trajectory  with  z(0)  G (0,  d(a  - Cr))  at  time  t - a-Cr 
must  be  nonpositive  according  to  Equation  (4.91): 

x*(a-Cr)<0,  for  x(0)  G (0, d(a  — Cr))-  (4.92) 

From  Equations  (4.90)  and  (4.92),  we  can  now  conclude  that  a Type-2  Structure  is  not 
optimal:  Let  u(t)  be  a production  policy  with  switching  time  t0.  According  to  Equation 
(4.90),  we  conclude  to  < (<  a — Cr)  and  x(to)  > 0.  This  implies 

x(a  ~ Cr)  = x(0)  -dt0  + (U-  d)(a  - Cr  - t0)  > x(0)  - dt0  = x(t0)  > 0.  (4.93) 

Since  Equation  (4.93)  violates  Equation  (4.92),  we  cannot  have  a Type-2  Structure. 

In  conclusion,  we  must  have  a Type-1  Structure,  i.e.,  the  optimal  trajectory  has  a 
singular  interval.  The  left  endpoint  satisfies  h - ^ at  which  the  trajectory  x*(t)  meets 
with  Trajectory  x^t).  According  to  the  Optimality  Condition  (see  Equation  (4.51)),  the 
trajectories  £*(f)  and  Xo(t)  will  be  identical  from  then  on.  That  is,  the  right  endpoint  of 
the  singular  interval  is  t2  = a — CT- 
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Thus,  the  optimal  control  has  the  singular  interval  - Cr\  and  satisfies 

o,  if<e[o,£M), 

“*(*)=  d,  if  t G [^,a  - Cr), 

U,  if  t G [a  — Cr,a). 

The  optimal  trajectory  x*(t ) satisfies 

0, 


(4.94) 


x*(t)  = 


if(€  fl  — Cr], 


(4.95) 


(U  — d)(t  — (a  — CT)),  if  t G [a  - CT,  a], 

( U — d)Cr  — d(t  — a),  if  t G [a,  a + b ]. 


Concerning  the  function  p*(f),  we  must  have  p*(  ^^)  = 0 and  p*(^^)— p*(t)  = ~P*{t)  — 


£121 


ft  d -2 x*(r)dT.  Therefore,  we  have  for  t G [0, 


P 


g(0)  o:(0) 

( t ) = ^ 2 x*(r)dT  — ^ 2 d( 


z(0) 


.x(0) 

d 


£121 


T)dr  = ~d(^- - rf  \td  =d( 


x(0) 


- t ?■ 


d ' v d ' 11  "v  d 

The  remaining  interval  [2^,a]  results  in  the  same  expression  for  p*(f)  as  in  Cases  B1 
and  B2  (see  Equations  (4.48)  and  (4.54)),  since  the  trajectory  x*(t ) for  t G [a  — Cr,a  + 6] 
is  identical  to  the  optimal  trajectories  in  Cases  B1  and  B2.  In  summary,  we  have 

d(^-t)2,  if(€[0,£g»], 

P'W=i  0,  if(€(^,o-C,],  (4.96) 

— ( U — d)(t  — (a  - CV))2,  if  / G [a  — Cr , a]. 

The  boundary  condition  is  verified  in  the  same  way  as  in  Cases  B1  and  B2,  since  the 
expressions  for  x*(a)  and  p*(a)  are  identical  to  those  for  the  Cases  B1  and  B2.  The  functions 
u*(t),  x*(t),  and  p*(t)  are  illustrated  in  Figure  4.13.  This  completes  our  discussion  of  the 
case  z(0)  G (0, d(a  — Cr )). 

Characterization  of  the  optimal  solution 

We  now  summarize  Equations  (4.46),  (4.52),  (4.63),  (4.81),  (4.85),  and  (4.94).  The 
optimal  control  policy  uj(f)  for  the  one-cycle  problem  P\  is  governed  by 
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Figure  4.13.  Optimal  Solution  for  Case  B5:  x(0)  G (0 ,d(a  — Cr )). 
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<(<)  = - 


U, 

ifz(0)<  -{U  - d)(a  - CT), 

and  t G [0,  a); 

u, 

if  z(0)  G (—(U  — d)(a  — Cr),  0), 

and 

d, 

if  *(0)  G (—(17  — d)(a  — Cr),  0), 

and 

*€[-^,0  Cr), 

u, 

if  a:(0)  G (—(17  — d)(a  — Cr),0), 

and 

t G [a  - CT,a); 

d, 

if  x(0)  = 0,  and  <G[0,a- 

Cr), 

u, 

if  x(0)  = 0,  and  t£[a-Cr,a); 

0, 

if  x(0)  G (0,  d(a  — Cr)),  and 

< £ [o,4?h 

d, 

if  a;(0)  G (0,  d(a  — Cr)),  and 

t G a - Cr), 

u, 

if  z(0)  G (0,  d(a  — Cr)),  and 

t G [a 

cr  •, 

0, 

if  z(0)  G [d(a  — Cr),  d(a  + |)), 

and 

t G [0,a  - 5i(x(0))), 

U, 

if  x(0)  G [d(a  — Cr),  d(a  + |)), 

and 

t e[a-  5i(*(0)),a): 

0, 

if  ®(0)  > d(a  + |),  and  <G[0,a). 

(4.97) 

We  observe  that  we  have  a singular  interval  for  x(0)  G (-({/  - d)(a  - Cr),d{a  - Cr)), 
a switching  point  for  x(0)  € [d(a  — Cr),d{a-\-  |)),  maximum  production  during  [0,a]  for 
*(0)  < —(U  — d)(a  — Cr),  and  no  production  for  x(0)  > d(a  + |). 

Figure  4.14  depicts  some  optimal  trajectories:  For  an  initial  production  surplus  level 
x(0)  within  the  range  [—(U  — d)(a  — Cr),d(a.  — CT)\,  the  trajectory  will  reach  the  zero 
production  surplus  level  at  a time  t G [0,a  - CT\.  The  trajectory  will  then  stay  at  the  zero 
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Figure  4.14.  Optimal  Trajectory  for  Problem  P\. 

production  surplus  level  until  time  t — a — Cr.  For  t £ [a  — CT,a),  the  machine  is  operated  at 
capacity  before  the  maintenance  takes  place.  The  machine  is  shut  down  during  the  interval 
[a,  a + b). 

Since  all  optimal  controls  with  singular  interval  have  the  same  right  endpoint  t2  = a- Cr, 
we  have 


x*(a)  = (U  - d)Cr,  for  a:(0)  6 [~(U  — d)(a  — Cr),  d(a  — Cr)].  (4.98) 

If  the  initial  production  surplus  level  x(0)  is  less  than  -{U  - d)(a  - Cr),  the  machine 
produces  at  its  maximum  rate  during  its  time  of  operation  [0,a]. 

For  an  initial  production  surplus  level  x(0)  satisfying  x(0)  £ [d(a  - CT),d(a  + |)),  we 
have  a switching  point  at  time  t0  = a - 5i(a;(0)). 

For  an  initial  production  surplus  level  z(0)  satisfying  z(0)  > d(a  + |),  the  machine  is 
shut  down  during  the  entire  interval  [0,a]. 

We  now  examine  the  trajectories  which  have  a switching  point  more  closely.  In  order 
to  illustrate  the  control  structure,  we  define  a switching  curve  xs(t0)  at  which  an  optimal 
control  switches  from  a production  rate  of  0 to  U.  Before  the  switching  time  <0,  the  machine 
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has  been  shut  down  and  the  production  surplus  decreases  with  rate  d.  Therefore,  at  the 
switching  point,  we  have 


xs(to)  = z(0)  - dt0,  for  <0  G [a  - Cr,a].  (4.99) 

From  Equation  (4.86),  we  also  have 

<0  = a - 5i(*(0)).  (4.100) 

Combining  Equations  (4.99)  and  (4.100)  leads  to 

Si{xs(to ) + dt0 ) = a -t0,  for  t0  G [a  - Cr,a}.  (4.101) 

From  Equation  (4.77),  we  obtain 


Si(xs(to)  + dt0) 


xs(to)  + dt0-da+Ub  xs(t0)  + dt0  - da  + Ub  2b  b 

uTi + V ( WTd > + imid(a  + 2>  - xsM  - dto) 


— a — to. 


By  rearranging  the  terms  and  taking  the  square,  we  conclude 


xs(t0)  + dt0  — da  + Ub 
U + d 


26  6 

)2  + + 2^  “ xs(to)  ~ dt0) 


rxs(to)  + dt0  - da  + Ub  2 

- ( uTd  + a~*o) 

,xS(t0)  + dt0  - da  + Ub  2 xs(<o)  + dt0  - da  + Ub  2 

- ( WTd  } +2 uTd («-*>)  + («-*>)• 

The  term  ( xs{to)+^-da+Ub  y canceis  out  on  both  sides.  Rearranging  the  remaining 
terms  results  in 


xs(t0) 


Ub 2 


U — d 


(a  + b - t0),  for  t0  € [a  - Cr,  a] 


(4.102) 


2(a  b — to)  2 

Starting  from  an  initial  surplus  z(0)  G [d(a  - Cr),d(a+  |6)],  the  production  surplus 
x*(t ) decreases  until  the  trajectory  reaches  the  switching  curve,  i.e., 


x*(t0)  = xs(t0), 


for  to  G [a  — Cr,a ]. 
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The  production  rate  is  then  switched  to  the  maximum  level  and  x*(t)  increases  until  the 
end  of  the  uptime  interval. 

At  the  endpoints  of  the  interval  [a  — Cr,a],  the  switching  curve  satisfies 


Xs((l  C T ) — 


and 

xs{a) 


Ub 2 


U-d 


2(a  + b - (a  - Cr))  2 
1 


(a  + b - (a  - Cr )) 


2(6  + Cr) 

1 

2(6  + Cr) 

0, 


Ub2 


(Ub2-(U -d)(b  + Cr)2) 

u 


(Ub  - (U  - d) 


U -d 


n 


U-d 


2(a  T 6 — a)  2 
db 
~2 


(a  + 6 — a) 


Notice  also  that  the  switching  curve  xs(t ) for  t G (a  — Cr,a]  lies  above  the  trajectory 
Xg(t),  since 


X, s((l  Ur') 

and 

d / x 

sIs(1) 


> 


Xo(a  - Cr)  = 0, 


Ub 2 U-d 

2(a  + b-t)2+  2 

Ub 2 U-d 

2(a  + b - (a  - Cr))2  + 2 

Ub 2 U-d 

2(6  + Cr)2  + 2 

U-d  U-d 
2 + 2 

(t/-d) 

^®Q(t),  for  t £ (a  — Cr,a). 


(4.103) 


(4.104) 


This  can  also  be  seen  in  Figure  4.14.  Thus,  the  optimal  trajectory  for  an  initial  production 
surplus  level  a:(0)  6 (d(a  - Cr),d(a  + |))  does  not  touch  the  trajectory  x^t). 
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Since  the  optimal  production  policy  ul(t)  is  determined  by  the  initial  production  surplus 
level  x(0),  we  can  therefore  express  the  optimal  performance  measure  in  dependence  of  x(0) 
as  well.  The  optimal  performance  measure  Jj  (x(0))  satisfies 


/“MO)  + (U-  d)t}2dt  + /a“+6{z(0)  + Ua-  dt}2dt , 

for  x(0)  < -{U  - d)(a  - Cr), 

=*121 

foU-d  {*(0)  + (U-  d)t}2dt  + - d)t}2dt 

+ foUU  ~ d)Cr  - dt}2dt, 

for  x(0)  G [—(U  — d)(a  - Cr ),  0], 


Jor{(U  - d)t}2dt  + Jq{(U  - d)Cr  - dt}2dt, 

for  x(0)  = 0, 


x(0) 

f0  d {*(0)  - dt}2dt  + /0Cr{(t/  - d)t}2dt  + /06{(t/  - d)Cr  - dt}2dt, 

for  a:(0)  G [0,  d(a  - Cr)], 


fZ~S'{xm{X(0)  ~ dt}2dt 

+ /oSl^*°)){x(0)  - d(a  - 5i(x(0)))  + (U  - d)t}2dt 
+ /“ +6{*(0)  + USi(x(0))  - dt}2dt, 

for  x(0)  G [d(a  — Cr ),  d(a  + |)], 


/0“+6{x(0)  - dt}2dt, 


for  z(0)  > d(a  + |). 


(4.105) 


110 


The  first  derivative  of  the  function  i]*  (x(0))  with  respect  to  *(0)  according  to  Appendix 
C.4  is 

2*(0)(o  + b)  + {U-  d)(a 2 + 2 ab  + ( b 2 - b2))  - db2, 

for  z(0)  < -(U  - d)(a  - Cr), 


e!M 
' U-d  ’ 


for  x(0)  € (—(U  — d)(a  — Cr),  0), 


4sjJrwo)H  ^ 


for  z(0)  € (0,  d(a  - Cr)), 


2®(0)(o  + b)  - d(a  + b)2  + f7S2(z(0))  + 2£fS1(s(0))6 

for  z(0)  G (d(a  - Cr),  d(a  + |)), 


2®(0)(a  + b)  - d(a  + b)2,  for  z(0)  > d(a  + |). 
In  Appendix  C.7,  we  verify  the  following  property: 


(4.106) 


dx( 0)^1  (*o)  < 0, 

for  x(0)  < 0, 

dx[ Oj^l  (X°)  ~ 

for  x(0)  = 0, 

(4.107) 

dx( 0)^1  (x°)  > O’ 

for  x(0)  > 0. 

From  Equation  (4.107),  we  conclude 

jr(xi(o))  < jr(x2(o)) 
^(*!(0))  < ^*(*2(0)) 

for  0 < zi(0)  < z2(0), 
for  a:2(0)  < ^(O)  < 0. 

(4.108) 

Ill 


The  second  derivative  of  Jx*(x(0))  with  respect  to  z(0)  according  to  Appendix  C.6  sat- 
isfies 


d 2 


2 (a  + 6), 

for  x(0)  < -(U  - d)(a  - Cr), 

-2x(0) 
U-d.  ’ 

for  z(0)  G (~(U  — d)(a  — Cr),  0), 

237(0) 

d ’ 

for  x(0)  G (0,  d(a  - Cr )), 

dx2( 0) 


J*i(M  o))  = 


rfj{a-5a(x(0))-a;(0)4uy51(a;(0)) 

+d(a  + 6)(jf  + 5i(x(0)))}, 

for  x(0)  e ( d(a  - Cr),  d(a  -f  |)), 


(4.109) 


2(a  + 6),  for  x(0)  > d(a  + |). 

The  second  derivative  is  positive  within  all  ranges.  By  examining  all  points  of  disconti- 
nuity of  the  second  derivative,  it  can  be  shown  that  the  function  (a;(0))  is  convex. 

This  concludes  our  analysis  of  Problem  Px.  In  the  next  section,  we  will  examine  a 
two-cycle  problem  before  we  solve  the  general  problem  with  N cycles. 


4.4  A Two-Cycle  Problem  and  its  Optimal  Solution 


In  the  previous  section,  we  solved  the  one-cycle  problem  Pi.  In  this  section,  we  extend 
those  results  to  the  two-cycle  problem.  The  time  interval  for  the  two-cycle  problem  is 
[0,2(a  + b)).  For  i € (a,a+  b ),  since  the  machine  is  undergoing  maintenance,  there  is  no 
control  decision  to  make,  i.e.,  we  have  u(i)  — 0 for  t G [a,  a + b).  According  to  the  Principle 
of  Optimality,  we  can  apply  the  result  for  the  one-cycle  problem  to  control  the  production 
rate  during  the  second  cycle  [a  + b,  2 (a  + 6)).  Therefore,  it  remains  to  consider  the  interval 
[0,a).  For  the  cost  during  the  interval  [a,2(a+  b)],  we  introduce  the  terminal  cost  h2(x(a)) 
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Figure  4.15.  A trajectory  for  the  two-cycle  problem 

at  t = a by  defining 

h2(x(a))  = h1(x(a ))  + Ji(x(a)  - db ),  (4.110) 

where  hi(x(a))  is  the  terminal  cost  for  Problem  jPi  with  production  surplus  x(a)  and 
J\  (x(a)  ~ db)  is  the  optimal  cost  for  a one-cycle  problem  starting  with  a production  surplus 
of  x(a)  — db. 

The  motivation  for  the  definition  of  h2(x(a))  is  illustrated  in  Figure  4.15.  The  term 
hi(x(a))  reflects  the  cost  during  the  interval  [a,  a + b]  and  the  term  Jf(x(a)  - db)  reflects 
the  cost  during  the  second  cycle,  i.e.,  the  interval  [a  + b,  2a  + 2b]. 

Notice  that  h2(x(a))  satisfies 

= 2z(a)&  - db2  + ^-^J*(x(a)  - db).  (4.111) 

The  control  problem  for  the  two-cycle  case  can  be  restated  as  follows. 


Problem  P2:  Determine  an  admissible  control  u*(t),  i.e.,  u*(t)  6 [0,  U],  for  t e [0,a],  that 
causes  the  system 


dx(t) 


= u(t)  — d 


dt 
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to  follow  a trajectory  x*(t)  which  minimizes  the  performance  measure 


i{u)  = [ x2(t)dt  + h2(x(a)). 
Jo 


The  boundary  conditions  are  z(0)  = x0  and  x(a)  free. 

For  Problem  P2,  the  Hamiltonian  (see  Pontryagin  [86],  p.  18  and  Kirk  [67],  p.  188)  is 
given  by 


7 i(x(t),u(t),p(t))  = x2(t)  + p(t)[u(t)  - d ], 


for  t £ [0,  a], 


where  the  function  p(t)  is  referred  to  as  costate  function. 

According  to  Pontryagin’s  Minimum  Principle,  necessary  conditions  for  an  optimal  con 
trol  law  u*(t)  for  Problem  P2  (see  Pontryagin  [86],  p.  69  and  Kirk  [67],  p.  233)  are 

dx*(t) 


dt 

dp*(t) 

dt 


= u*(t)  — d\ 
= -2  x*(t); 


(4.112) 

(4.113) 


x*2(t)  + p*(t)[u*(t)  - d]  = min  {x*2(t)  + p*(t)[u(t)  - d]} 

u(t)e[o,t/] 


x*  (t)  - p*(t)d+  min  (p*(t)u(t)}.  (4.114) 


The  boundary  condition  for  Problem  P2  satisfies 


d 


2 x*(a)b  - db 2 - p*(a)  + ---  J^(x(a)  - db)  = 0. 

CLX  yCL) 


(4.115) 


Since  the  final  time  is  fixed  and  the  Hamiltonian  does  not  explicitely  depend  on  time, 
we  have 

x*2(t)  + p*(t)[u*(t)  - d]  = ci,  for  t £ [0,  a}.  (4.116) 

From  Condition  (4.114),  the  optimal  control  satisfies 


u*(t)  = i 


0,  ifp*(t)>0, 

U,  if  p*(t)  < 0, 

/?,  if  P*(t)  = 0. 


(4.117) 


where  (3  £ [0,  U]  is  to  be  determined. 


114 


According  to  Equation  (4.117),  the  optimal  control  law  u*(t)  is  thus  determined  when- 
ever we  have  p*(t)  ^ 0.  In  the  case  that  roots  exist  for  the  function  p*(t)  within  the  interval 
[0,g],  in  the  same  way  as  in  Sections  4.3.1  and  4.3.2  for  Problem  Pi,  we  further  distinguish 
between  singular  intervals , switching  times,  and  switching  points.  For  Problem  P2,  virtu- 
ally the  same  analysis  as  for  Problem  P\  leads  to  the  same  results  with  respect  to  singular 
intervals  and  switching  points. 

4.4.1  Singular  Intervals  for  Problem  P> 

For  Problem  Pi,  we  concluded  that  if  there  is  a singular  interval  within  [0,4  there  is 
neither  a second  singular  interval  nor  a switching  point  within  [0,a].  This  result  is  based 
on  Equation  (4.16).  It  is  independent  of  the  terminal  cost.  The  corresponding  Equation 
for  Problem  P2  is  Equation  (4.116).  We  conclude  that  for  Problem  P2,  if  there  is  a singular 
interval  within  [0,  a],  there  is  neither  a second  singular  interval  nor  a switching  point  within 

[0,4 

Next,  we  demonstrate  that  Property  (4.26)  for  Problem  Pi  can  be  verified  for  Problem 
P2  as  well. 

If  two  admissible  controls  u\t ) and  u"(t)  lead  to  trajectories  x\t ) and  x"(t) 
such  that  (a)  |*'(t)|  < \x"(t)\,  for  t 6 [0,4  and 
(b)  either  0 > x\a)  > x"(a), 
or  db  < x\a)  < x"(a), 

then,  we  have  J2(u ')  < J2{u'').  (4.118) 

To  see  the  validity  of  this  property,  notice  the  following: 

1.  We  have  /0a(a:'(f))2dl  < /“( x'\t))2dt , since  |z'(f)|  < \x"(t)\  for  t 6 [0,4 

2.  We  have  /06(x'(a)  - dtfdt  < fo(x"(a)  - dtfdt,  for  0 > x\x)  > x"(a)  or  x"(a)  > 
x'{a)  > db. 
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3.  By  Equation  (4.108),  for  0 > x'(a)  > x"(a ) (i.e.,  0 > - db  > x'(a)  - db  > x"(a)  - db), 
or  x"(a)  > x\a)  > da  (i.e.,  0 < x'{a ) — db  < x'\a ) — db),  we  have  J{ (x'(a)  - db)  < 
Ji(x"(a)  - db). 


Therefore,  we  can  conclude 


J2(u')  = f x,2(t)dt  + [ (x'(a)  - dt)2dt  + J{(x'(a)  - db) 

Jo  Jo 

< r x"2{t)dt  + l\x"(a)  - dtfdt  + Ji(x"{a)  - db)  = J2(u"). 

Jo  Jo 

In  the  same  way  that  we  use  Property  (4.26)  to  conclude  Equation  (4.27)  for  Problem  Pt, 
we  use  Property  (4.118)  to  conclude  for  Problem  P2, 


u*(t)  — < 


01, 

d, 

U, 


if  t € [0,ii), 
if  t € [<1,^2)) 
if  t € [t2,a), 


(4.119) 


where  G (0, 17} . 

4.4.2  Switching  Points  for  Problem  P9 

Based  on  Property  (4.118),  we  can  establish  that  if  the  optimal  control  has  a switching 
point  x*(to)  at  time  to,  we  must  have 


x*(to)  > 0, 

(4.120) 

z(0)  > 0, 

(4.121) 

u*(t)  = < 

' 

0,  if  t G [0,  to), 

(4.122) 

U,  ifte[<o,a)- 

Concerning  the  terms  singular  interval  and  switching  point,  please  notice  that  in  our 


analysis,  these  terms  are  only  used  in  reference  to  the  interval  [0,a].  The  remaining  time 
interval  can  be  solved  according  to  the  Principle  of  Optimality  and  the  solution  of  Problem 

Pi- 
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4.4.3  Initial  Control  Level  for  Problem  P? 


In  Section  4.3.3,  we  have  determined  the  initial  control  level  for  Problem  P\.  We  have 
done  so  by  showing  that  two  of  the  three  potentially  optimal  control  levels  0,  d,  and  U 
cannot  be  optimal  for  certain  initial  production  surplus  values.  We  have  distinguished  six 
cases  A1  to  A6  (see  Table  4.2  in  Section  4.3.3).  We  use  the  same  cases  for  our  analysis  of 
the  initial  control  level  for  Problem  P2. 

In  the  following,  we  explain  how  we  arrive  at  the  same  result  for  Problem  P2  as  for 
Problem  P\. 

To  show  that  Cases  A1  and  A6  are  not  optimal  for  Problem  Pi,  we  have  used  Property 
(4.24).  Since  Property  (4.24)  is  independent  of  the  terminal  cost,  it  holds  for  Problem  P2 
as  well.  Therefore,  for  Problem  P2,  Cases  A1  and  A6  are  not  optimal  either. 

For  Problem  Pi,  Cases  A2  and  A4  are  based  on  Property  (4.26).  For  Problem  P2,  we 
have  shown  the  corresponding  Property  (4.118)  above  (see  Section  (4.4.1)).  We  therefore 
conclude  that  Cases  A2  and  A4  for  Problem  P2  are  not  optimal  either. 

For  Cases  A3-1  and  A5  in  Problem  Pi,  we  also  use  Property  (4.26).  In  the  same  way, 
the  results  are  transferred  for  Problem  P2.  The  remaining  subcase  (Case  A3-2)  needs  to 
be  modified  for  Problem  P2,  since  Problems  Pi  and  P2  have  different  terminal  cost.  The 
calculations  for  Case  A3-2  for  Problem  P2  are  shown  in  Appendix  C.9.  The  result  is  the 
same  as  in  Section  4.3.3  for  Problem  Pi. 

In  summary,  for  Problem  P2,  the  initial  control  level  satisfies 


u*(0)  = - 


4.4.4  Optimal  Solution  for  Problem  P? 


0, 

d, 

U, 


if  x(0)  > 0, 
if  x(0)  = 0, 
if  z(0)  < 0. 


(4.123) 


In  this  section,  we  derive  the  optimal  control  in  dependence  of  the  initial  production 


surplus  x(0)  based  on  the  observations  in  the  previous  sections. 
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Due  to  the  different  terminal  cost,  we  partition  the  range  of  a; (0)  for  Problem  P2  in  a 
slightly  different  fashion: 


Cl:  x(0)  = 0, 

C2:  *(0)€(-(tr-d)(o-|^),0), 

C3:  x(0  )<-{U-d){a-l-j^-L), 

C3-1:  *(0)  G [-(U  - d)a , -(U  - d)(a  - 

C3-2:  *(0  )<-(U-d)a, 

C4:  .(0)>«.-ig£j), 

C4-1:  x(0)  > d(a  + 6), 

C4-2:  *(0  )e[d(a-±-^),d(a  + b)), 

C4-2-1:  x(0)  G (da  + (U  - sju(U  - d))6,  d(a  + b )), 

C4-2-2:  *(0)  G [d(a-  + (P  - ^(P  - d))b], 

C5:  s(0)e(0,d(a-i^)). 

As  we  have  seen  for  Problem  Pi,  the  three  possible  structures  for  the  optimal  control  are  a 
Type-1  Structure,  a Type-2  Structure,  and  a Type-3  Structure.  For  Problem  P2  and  each 
x(0)  value,  the  optimal  control  has  one  of  these  structures  as  well. 

Case  Cl:  a:(0)  = 0 


According  to  Equation  (4.123),  we  have  u*(0)  = d.  Due  to  Properties  (4.34)  and  (4.36), 
this  excludes  a Type-2  Structure  and  a Type-3  Structure  from  consideration.  Consequently, 
the  optimal  trajectory  has  a Type-1  Structure  which  has  a singular  interval  [fi,t2]-  Since 
u*(0)  = d,  the  singular  interval  has  the  left  endpoint  t\  = 0.  For  the  right  endpoint,  we 
introduce  parameter  C = a-<2,  so  that  we  have  [h,t2]  = [0,a-C].  From  Equation  (4.119), 
we  obtain  for  the  production  control 


u(t)  = < 


d, 

V, 


if  t £ [0,a  - C ), 
if  t 6 [a  - C,  a), 


(4.124) 
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and  for  the  trajectory 


x(t)  = 


0, 

(U  — d)(t  — (a  — C)), 
(U  - d)C  - d(t  - a), 


if  t e [0,a  - C), 
if  t e [a  - C,  a), 
if  t £ [a,  a + b), 


The  performance  measure,  expressed  in  terms  of  C,  is 


T2(C)  = / x2{t)dt  + h2(x(a)) 

Jo 

C b 

= I ((U  - d)t)2dt  + / ((£f  - d)C  - dtfdt  + J*((C7  - d)C  - db).  (4.125) 
Jo  Jo 

We  seek  to  determine  the  value  of  C G [0,a)  which  solves 


min{  J2(C)  : Ce[0,a)}. 


(4.126) 


We  differentiate  J2(C)  by  using  Leipniz’  rule  and  obtain 

~J2(C)  = ((U-d)C)2+  / 2 (U-d)((U-d)C-dt)dt  + ~J*((U-d)C-db) 

a{ ^ Jo  dC 

= (u  - d)2{C 2 + 2bC  - - d)C  - db).  (4.127) 

In  order  to  solve  (4.126),  we  first  examine  the  roots  of  the  derivative  ^J2(C)  within  the 
range  [0,a).  We  consider  each  term  in  Equation  (4.127)  separately. 

The  first  term  in  Equation  (4.127)  is  identical  to  -^J\(C)  for  the  one-cycle  problem 
with  z(0)  = 0.  According  to  Equations  (4.41)  and  (4.60),  we  have 


C2  + 2 bC  - ^b2  < 0,  for  C G [0,  Cr), 

C 2 + 2bC  - jj^b2  > 0,  for  C G (Cr,  a), 


(4.128) 


where  CT  = b(\fj^d  ~ 1)  *s  the  same  constant  which  optimizes  (4.40). 

For  the  second  term  in  Equation  (4.127),  we  use  Property  (4.107)  and  conclude 


for  C < 

for  C > r&d. 


J&Ji((U  — d)C  — db)  < 0, 

&j;((u-d)c-db)>  o, 


(4.129) 


Since 
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CV  = b(\ 


U 


lU-d ^ = b (VU)2  - {y/U^df  < 1 db 


U-d  U — d y/U~^d(VU  + VU^d)  2U-d ’ 
and  db  < (U  — d)a,  we  have 


0 < CT  < 


db 

U-d 


< a. 


(4.130) 


(4.131) 


Therefore,  with  Equation  (4.131),  we  can  conclude  from  Equations  (4.127),  (4.128)  and 
(4.129) 


JoMC)  < 0)  f°r  C € [0,Cr), 

(C)  > 0,  for  C € (ifzd,a). 


(4.132) 


This  implies 


min{J2(C)  : C€[0,Cr]}  = J2(Cr), 

min {J2(C):  Ce[^,«)}  = J2{fy. 

Consequently,  the  solution  for  (4.126)  must  be  within  the  interval  [Cr,  jj^].  Since  we  have 


(U  - d)Cr  -db>{U-  d)Cr  -(U-  d)a  = -( U - d)(a  - CT),  (4.133) 

C € [Cr,  implies  ( U - d)C  - db  G [(U  - d)Cr  - db , 0]  C [-([/  - d){a  - Cr ),  0]. 

(4.134) 

According  to  Equation  (4.106),  we  have 

d xP" ( 

Jr(x(0))  = for  x(0)  e (-(tf  - d)(a  - Cr),  0).  (4.135) 

Applying  the  chain  rule,  we  therefore  obtain  for  C € [Cr, 

^i‘((C - d)C -<»)  = ((/-  d){-aU  ~ W ~ db? } = -((/-  ^(C  _ (4.i36) 


Substituting  Equation  (4.136)  into  Equation  (4.127),  we  obtain 


d 

dC 


MC ) 


d 


(C- 


db 


f)2} 


(U  - d)2{C2  + 2bC  ~ n _ v_d, 

(V  - df{ 26(1  + ^)C -6(1  + 


U-d' 

mu -me- 


U-d'U-d 


db 


U-d 


], 


(4.137) 


for  C e [Cr, 


and 
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^ J2(C)  = 2 U(U  - d)b,  for  C e [Cr,  j^\. 
Therefore,  we  have  a unique  minimum  at  C — since 

^7J2(C)  = 0 if  and  only  if  C = 


and 

16*^211  - d)  = 2U(U~d)b  > °- 

Thus,  C = \ifzd  s°lves  (4.126).  This  means  that  the  optimal  control  has  the  singular 
interval  [0,o  - produces  at  maximum  during  [a  - and  undergoes  main- 

tenance during  [a,  a + b].  At  the  end  of  the  first  cycle,  the  production  surplus  level  is 
x*(a  + b)  = db . 

According  to  the  Principle  of  Optimality,  the  optimal  trajectory  during  [a  + 6,2(a  + 6)] 
for  Problem  P2  can  be  obtained  using  the  optimal  control  for  Problem  Pi  emanating  from 
x(a+b).  Therefore,  during  [a+6, 2a+26],  the  production  is  controlled  according  to  Equation 
(4.97)  for  an  initial  surplus  of  — ^ db . Thus,  it  reaches  the  zero  production  surplus  level  again 
at  t — a + b + From  db  < (U  — d)a  and  Equation  (4.130),  we  conclude 


1 db  ^ 1 db 

2 U -d~a~ 2 U-d 


< a — Cr. 


(4.138) 


Thus,  the  second  cycle  has  the  singular  interval  [a  -f  b + -f  b — Cr\.  Within 

[2a  + b-  Cr,  2 a + 6],  we  produce  at  maximum  capacity,  and  within  [2a  + 6, 2a  + 26],  we  have 
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another  maintenance  interval.  In  summary,  we  have  for  the  control  law 


d, 

U, 

0, 


u*(t)  = < 


U, 

d, 

U, 


0, 


iftG[0,a-i^), 

if  t £ [a  — 2 a)? 

if  t £ [a,  a + 6), 

if  1 € [a  -|-  6,  a T 6 T ^ db ), 
if  t £ [a  -(-  6 -j-  T^db,  2a  -\-  b — Cr), 
if  t £ [2a  + 6 - Cr,  2a  + 6), 
if  1 £ [2a  + b,2a  + 26). 


(4.139) 


The  optimal  trajectory  £*(/)  satisfies 


0, 

(C- <*)((- (a 

^<16  — d(t  — a), 


if  t £ [0,  a 2 
if  t 6 [a  — 2 {jrj  ? a]  > 

if  1 6 [a,  a + 6], 


**(*) 


(17  - d)(t  - (a  + b ))  - |d6, 

0, 

(17  - d)(t  - (2a  + b - Cr )), 
(17  - d)CT  - d(t  - (2a  + 6)), 


if  t £ [a  + b,  a + b + 2 
if  t G [n  + b + 2 17^3’  2a  + 6 — Cr], 
if  t £ [2a  + b — Cr,  2a  + 6], 
if  1 £ [2a  + 6,2a  + 26], 


(4.140) 


We  have  included  the  second  cycle  in  the  expressions  for  u*(t)  and  x*(t)  for  illustra- 
tive purposes.  For  the  remaining  cases  C2,  C3,  C4,  and  C5,  we  simply  give  the  optimal 
production  control  and  the  optimal  trajectory  for  the  first  cycle,  since  the  second  cycle 
for  x(a  + 6)  = x'  of  the  two-cycle  problem  is  controlled  in  the  same  way  as  the  one-cycle 
problem  for  x(0)  = x' . 
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We  now  analyze  the  costate  function.  For  the  costate  function  p*(t ),  we  have  p*(t)  = 0 
for  t G [0,  a - Using  Equation  (4.113),  we  have  for  t G [a  — a], 

1 db 


p‘(t)  = f(t)-p-(a-\JL2)  = -2 


2 U-d 

1 db 


*'  u <-»  ri  j 

i 

In  summary,  we  have 


2 C7-d 

1 db 


a 


2 U-d 


))dr 


4 (“  2 U-d)  1 db 

= -2  / (t7  - d)TdT  = -( U - d)(i  - (a  - -^— ^)) 2. 


P*(0  = < 


°,  if  t G [0,a- 

, -(ff -<*)(*- («-§*))*,  if<€[a-i^,a]. 

We  verify  the  boundary  condition  (see  Equation  (4.115)): 

2x*(a)b  - db 2 - p*(a)  + ^*(**(0)  - rf6) 


(4.141) 


dx{a ) 

= 0. 


(4.142) 


This  concludes  the  analysis  for  Case  Cl  where  x(0)  = 0.  As  for  the  one-cycle  problem, 
before  we  proceed  with  the  cases  where  the  initial  production  surplus  is  negative  or  positive, 
we  state  an  optimality  condition. 

Optimality  Condition 

If  a;^/)  and  x^t)  are  two  optimal  trajectories  for  Problem  P2  with  initial  production 
surplusses  ^(O)  and  x^O),  respectively,  then  the  trajectories  will  not  cross  each  other,  i.e., 

if  x\(t')  < *2(0  t°r  some  t'  G [0,a],  then  xl(t)  < x2(t)  for  all  t G [0,a],  (4.143) 


and  in  particular  for  t'  — 0, 


if  aq(0)  < x2(0), 


then 


^i(t)  < *2(0>  f°r  t G [0,a]. 


(4.144) 
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Case  C2:  z(0)  G (-(V  - d)(a  - iffi).  0) 

First,  since  x(0)  < 0,  the  initial  control  level  satisfies  u*(0)  = U according  to  Equation 
(4.123).  We  now  examine  the  control  structure.  Since  u*(0)  = U,  we  exclude  a Type-2 
Structure  according  to  Equation  (4.34).  The  optimal  trajectory  for  z(0)  G (-(U  - d)(a  - 
5t7Zd)>0)  meets  the  trajectory  x^(t)  at  time  t = According  to  the  Principle  of 

Optimality,  these  trajectories  cannot  cross  each  other  and  we  conclude  that  the  optimal 
control  has  the  singular  interval  [ , & — \ipZd\-  Therefore,  we  have  for  the  optimal 

control  u*(t) 


(4.145) 


The  optimal  trajectory  x*(t)  satisfies 


(V  - d)(t  - 


0, 


(4.146) 


For  the  costate  function,  we  obtain  for  t G [0,-^^]  according  to  Equation  (4.113), 


(4.147) 


Since  the  trajectory  x*(t)  for  t G [ -fi-d' ’ a + is  identical  to  the  trajectory  in  Equation 
(4.140),  the  remaining  interval  [ , a]  yields  the  same  expression  for  p*(t)  as  in  Equation 
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(4.141).  In  summary,  we  have 


P*(!)  = < 


0, 


Since  the  expressions  for  x*(a)  and  p*(a)  are 
condition  (see  Equation  (4.115))  is  verified  in  the 
concludes  the  considerations  for  the  Case  C2  with 


(4.148) 

if  t £ [a  — 2U^dia]- 

the  same  as  in  Case  Cl,  the  boundary 
same  way  as  in  Equation  (4.142).  This 
*(0)  €(-(tf- <Q(a- life),  0). 


Case  C3:  adOl  < -(V  - d)(a  - 


First,  since  a;(0)  < 0,  the  initial  control  level  satisfies  w*(0)  = U according  to  Equation 
(4.123).  Next,  we  examine  the  control  structure.  According  to  Property  4.34,  we  can  exclude 
a Type-2  Structure  (a  production  control  with  a switching  point),  since  it  requires  u*( 0)  = 0. 
Therefore,  the  optimal  control  has  either  a Type-1  Structure  or  a Type-3  Structure.  That  is, 
the  optimal  control  either  produces  at  maximum  capacity  or  it  has  a singular  interval.  By 
distinguishing  between  the  cases  z(0)  £ [-(U  - d)a , -(U  - d)(a  - \ jjzj)]  (Case  C3-1)  and 
x(0)  < —(U  — d)a  (Case  C3-2),  we  show  that  the  optimal  control  has  a Type-3  Structure. 

Case  C3-1:  *(0)  € [-(U  - d)a, -{V  - d){a  - ±^)]. 

If  we  have  a Type-3  Structure,  we  produce  at  maximum  capacity , 


u(t ) = U,  for  t £ [0,  a). 


(4.149) 


If  we  have  a Type-1  Structure,  we  have  singular  interval  [<!,<2]  and  the  production  control 
satisfies  Equation  (4.119).  Since  we  have  u*(0)  = U,  we  conclude  - U in  Equation 
(4.119).  The  left  endpoint  of  the  singular  interval  satisfies  t\  = ■ Let  t2  = a — C where 

C £ [0,a  — With  [<i,i2]  = , a — C],  the  production  control  thus  satisfies 


u(t)  = < 


U, 

d, 


if  f e [0?  u-d  )’ 
if  <€  [^,a-C), 


(4.150) 


U,  if  t £ [a  — C,a). 
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Notice  that  we  can  represent  the  production  control  in  Equation  (4.149),  by  letting 
C = a — in  Equation  (4.154).  Thus,  if  we  extend  the  range  of  C from  [0,  a — ) to 

[0> a TJ  in  Equation  (4.150),  we  have  represented  both  control  structures  which  need 
to  be  considered. 

We  can  express  the  performance  measure  as  a function  of  the  parameter  C, 

J2(C)  = [ U~d  ((£/■  - d)tfdt  + / ((U-d)t)2dt  + ( {{U -d)C -dtfdt 

Jo  Jo  Jo 

+Ji({U  - d)C  - db).  (4.151) 


We  seek  to  determine  the  value  of  C which  solves 


min{J2(C')  : C € [0,a  - 


-»(Q) 

U - d 


]}• 


(4.152) 


Notice  that  the  cost  function  J2(C)  in  Equation  (4.151)  differs  from  the  cost  function 
in  Equation  (4.125)  only  by  the  term  f0u~d  ((U  - d)t)2dt.  This  term  is  independent  of 
C.  Therefore,  the  derivatives  of  the  cost  functions  in  Equations  (4.151)  and  (4.125)  with 
respect  to  C are  identical  (see  Equation  (4.127)): 


±MO  = (u-inc'  + nc-^ 


With  the  same  reasoning  as  in  Case  Cl,  we  obtain 

< 0 


A 

dC 


h(C)  = 


} + acJi((u-d)c-db). 


for  C 6 [0,Cr), 


2U(U-d)b(C-  forCG[Cr, 


db  l 
U-d  b 


(4.153) 


> 0 


for  C€(A,4 


Since  we  have 


— a:f0)  1 

a — „ < a + — -(  — ([/  — d)(a  — ^ 


1 db 


U-d 


U-d ' 
1 db 


2 U-d 


)) 


2 U-d' 

we  conclude  from  Equation  (4.153) 

d 


for  x(0)  < —{U  - d)(a  - 


1 db 


2 U -d‘ 


-x(0) 


dcMc)<0,  force  [0,0  u_d 


), 
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and  thus,  C = a — solves  (4.152).  Equation  (4.150)  with  C = a — jjr^i  leads  to 

u*{t)  = U,  fortG[0,a).  (4.154) 


Therefore,  we  have  shown  that  the  optimal  control  for  z(0)  G [—(U—d)a,  —(U—d)(a—^jj^)\ 
is  a Type-3  Structure. 

Case  C3-2:  x(0)  < —{U  — d)a. 

Recall  that  we  have  already  excluded  a Type-2  Structure.  In  this  case,  we  can  also  exclude 
a Type-1  Structure,  since  the  trajectory  x(t)  cannot  reach  the  zero  production  surplus  level 
for  t G [0,a]  and  ®(0)  < -(U  - d)a. 

Thus,  we  have  a Type-3  Structure,  and  since  the  initial  control  level  satisfies  u*(0)  = U, 
we  have  the  same  optimal  control  as  in  Case  C3-1  (Equation  (4.154)). 

For  the  optimal  trajectory  in  both  Cases  C3-1  and  C3-2,  we  have 


x*{t)  = < 


(4.155) 


z(0)  -f  (U  — d)t,  if  t G [0,  a], 

x(0)  -f  (U  — d)a  — d(t  — a),  if  t G [a,  a + b ]. 

For  the  costate  function,  we  use  Equation  (4.113),  to  obtain  p*(a)—p*(t)  = /“  — 2a:*(r)dr, 
or  equivalently, 


P*(t)  = 2 J x*(r)dr  + p*(a), 

= 2j\x(0)  + (U-d)T)dT  + p*(a), 

— 2x(0)(a  - t)  + (U  - d)(a2  - t2)  + p*(a),  fortG[0,a].  (4.156) 


The  boundary  condition  (Equation  (4.115))  results  in  an  expression  for  p*(a ): 

p*(a)  = 2 x*(a)b  - db2  + ----  ■■  Jf(x*(a)  - db ),  (4.157) 

dx(a) 

where  (x*(a)  — db)  is  given  by  Equation  (4.106). 

Together  with  Equations  (4.156)  and  (4.157),  we  obtain 


d 

dx(a) 


p*(t ) = 2 (a  + b — t)x(0)  + (U  — d)(a2  + 2ab  — t 2)  — db2  + 


Ji(xm(a)  - db).  (4.158) 
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This  completes  the  analysis  of  the  case  x(0)  < —({7  — d)(a  — \jpzd)- 
Cases  Cl,  C2,  and  C3  complete  the  analysis  of  x(0)  < 0.  In  the  following,  we  consider 
the  cases  where  x(0)  > 0. 

Case  C4:  ar(0)  > d(a  — 

First,  since  x(0)  > 0,  the  initial  control  level  satisfies  u*(0)  = 0 according  to  Equation 
(4.123).  Next,  with  similiar  reasoning  as  for  Case  B4  in  Section  4.3.4,  we  can  exclude  a 
Type-1  Structure  from  consideration. 

Thus,  we  have  either  a Type-2  Structure  or  a Type-3  Structure:  Since  we  have  w*(0)  = 0 
for  a:(0)  > 0 according  to  Equation  (4.123),  we  either  shut  the  machine  down  during  the 
entire  interval  or  we  switch  the  production  rate  from  0 to  U at  some  time  t0  G (0,a). 

If  we  shut  the  machine  down , the  control  policy  is 


u(t)  — 0,  for  t G [0,  a). 


(4.159) 


If  we  have  a switching  point  policy  with  a switching  time  t0  G (0,  a),  we  let  S = a - t0  G 
(0,u).  According  to  Equation  (4.122),  we  have  with  switching  time  /0  = a — 5, 


u(t)  = 


0,  if  t G [0,  a - 5), 

U,  if  t G [a  — S,  a). 


(4.160) 


We  now  combine  the  solution  structures  by  extending  the  range  of  S in  Equation  (4.160) 
from  (0,a)  to  [0,a),  since  the  production  policy  in  Equation  (4.159)  is  represented  by  the 
policy  in  Equation  (4.160)  with  5 = 0. 

In  order  to  obtain  the  performance  measure,  we  consider  the  two  cycles  separately.  For 
the  first  cycle,  we  obtain  the  same  expression  as  in  Case  B4  (see  Equation  (4.73)). 

Since  we  have 


x*(a  + b)  = z*(a)  -db  = x(0)  + US  - d(a  + b ) 
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the  cost  for  the  second  cycle  is  given  by  Ji(a:(0)  + US  - d(a  + &)).  Therefore,  the  cost 
function  J2(S ) satisfies 


h(S)  = / (x(0)  - dt)2dt  + / (z(0)  - d(a  - S)  + (U  - d)t)2dt 

Jo  Jo 

fb 

+ / (x(0)  - da  + US  - dtfdt  + J*(x(0)  + US  - d{a  + b )).  (4.161) 
Jo 


We  are  seeking  the  value  of  parameter  S which  solves 


min{J2(S)  : S G [0,a)}. 


(4.162) 


For  the  derivative,  we  have  according  to  Leipniz’  rule 

Aj2(5)  = -l(x(0)-d(a-S))2  + {x(0)-d(a-S)  + (U-d)S)2 

+ / 2d(x(0)  - d(a  - S)  + (U  - d)t)dt  + [b  2U(x(0)~  da  + US  -dt)dt 

Jo  Jo 

+ + US-  d(a  + b )). 


Evaluating  the  integrals  and  rearranging  the  terms,  we  obtain 


dS 


J2{S)  = U(U  + d){S2  + 2 


d 


x(0)  - da  + Ub 
U + d ' 


+ —j;(x(0)  + US-d(a  + b)). 


(4.163) 


In  the  following,  we  distinguish  the  cases  x(0)  > d(a  + b)  (Case  C4-1)  and  x(0)  G 
[d(a  — \ jfZd),  d(a  + 6))  (Case  C4-2). 

Case  C4-1:  a;(0)  > d(a  + b). 

We  have  according  to  Property  (4.107) 

— Jj  (x(0)  + US  — d(a  + b ))  > 0,  for  x(O)  > d(a  + b)  and  S G [0,  a). 


According  to  Equation  (4.163),  we  conclude 

J2(S)  > 0,  for  x(0)  > d(a  + b)  and  S G [0,  a). 
Consequently,  S = 0 solves  (4.162).  This  leads  to  the  control 


u*(t ) = 0, 


for  t G [0,  a), 


(4.164) 
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and  the  trajectory 

x*(t)  = x(0)  - dt,  for  t G [0,a  + 6].  (4.165) 

With  p*(a)  — 2 x*(a)b  - db 2 + {x(a)  - db ) from  the  boundary  condition  (see 

Equation  (4.115)),  Equation  (4.113)  for  t G [0,a]  leads  to 

P*(t)  = 2 J*  x*{T)dr  + p*(a) 

= 2 f (x(0)  - dr)dr  + 2(x(0)  - da)b  - db2  + — J?(x(a)  - db) 

Jt  dx[a) 

= 2*(0)(a  - t)  - d{a2  - t2)  + 2(x(0)  - da)b  - db2  + — J?(x(a)  - db) 

uX  yd) 

= 2x(0)(a  + b-t)-d{{a  + b)2 -t2)+  -A-j;(x(0)  - d{a  + b)).  (4.166) 

This  completes  our  analysis  for  x(0)  > d(a  + b ). 

Case  C4-2:  a:(0)  G [d(a  - d(a  + b )). 

In  order  to  evaluate  the  right-hand-side  expression  for  the  derivative  in  Equation  (4.163), 
we  need  to  evaluate  the  term  ^Jj(x(0)  + US  - d(a  + b)).  According  to  Equation  (4.106), 
we  therefore  need  to  identify  the  possible  range  of  x(a  + b)  = z(0)  + US  - d{a  + b ).  From 
Cases  Cl  and  C4-1,  we  have 

x*(a  + b)  = ~\db,  for  x(0)  = 0,  (4.167) 

x*(a  + b)  — 0,  for  ar(0)  = d(a  + b).  (4.168) 

Together  with  the  Optimality  Condition  (see  Equation  (4.144)),  we  conclude  from  Equa- 
tions (4.167)  and  (4.168) 

x(a  + 6)  = z(0)  + US  - d(a  + b)  G [-^d&,0],  for  x(0)  G [0,  d(a  + 6)].  (4.169) 

Since  we  have  -\db  > -([/  - d)(a  - Cr)  (see  Equation  (4.138)),  we  also  have 

[~db,  0}C  [-(U  - d)(a  - Cr),Q).  (4.170) 

From  Equations  (4.169)  and  (4.170),  we  obtain  the  following  expression  according  to  Equa- 
tion (4.106), 

(*(0)  + US  - d{a  + b))2 
U-d  ’ 


^JZ(x(0)  + US  -d(a  + b))  = U( 
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U 

U -d 


(*(0)  + US  - d(a  + b))2.  (4.171) 


We  can  now  substitute  Equation  (4.171)  into  Equation  (4.161)  and  obtain 


Aj2(5)  = u(U  + d){S2  + 2 


dS 


x(0)  - da  + Ub 
U + d 


5 ~ irh(d(a + b2]  ~ I(0))) 


U 


U -d 


(x(0)  + US  - d(a  + b))2. 


(4.172) 


According  to  Appendix  C.10,  this  equation  can  be  rewritten  as  follows: 
d Ud2  r„  2 r)U2b  — d{x(0)  - da) 


dsJ M = ~/s~2 


d 2 


, x2(0)  - 2(776  + da)s(0)  + d(da2  + 2Uab  + Ub2) , , t , 

+ }.  (4.173) 


Consequently,  the  roots  of  ^ J2(5)  are 


, _ U2b  — d(x(0)  — da)  U2b  — d(x(0)  — da) 


S',  = 


S"  = 


d2 


(- 


d2 


-)2-/(*(o)), 


U2b  — d(x( 0)  — da)  + / U2b  - d(x( 0)  - da) 


d2  'V'  d2 

where  f(x(0))  = x2(°')-2(Ub+da)x(°'>+d(da2+2Uab+Ub2) 

For  the  second  derivative,  we  have 


-)2  - /(*(<>)), 


<P  «2  V2b  - d(x(0)  - da) 

dsiMS)  - “ rrr~dl2S  ~ 2 # >' 


and  therefore, 


jg 2 *^2 (^2)  > 0, 

< 0. 


Since  we  are  minimizing,  we  focus  on  S'2  and  rename  it  as  52(x(0)): 
U2b  — d(x(0)  - da)  I U2b  — d(x(0)  — da) 


52(a:(0)) 


d2 


d2 


/(*(0)).  (4.174) 


The  unrestricted  minimum  is  therefore  at  S = 52(a:(0)): 


min{J2(5)  : S unrestricted  } = J2(52(a:(0))). 


(4.175) 
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Since  the  range  for  5 in  (4.162)  is  restricted  to  [0,  a),  we  distinguish  the  cases  S2(x(0))  < 
0 and  52(z(0))  > 0.  According  to  Equation  (C.69)  in  Appendix  C.10,  we  have  that  z(0)  G 
[d(a  - \ d{a  + 6))  and  52(a:(0))  < 0 imply 

*(0)  G {da  + {U-  yj U{U  - d))b,  d{a  + b )), 

and  :r(0)  G [d{a  - d(a  + b ))  and  52(a;(0))  > 0 imply 

x(0)  G [d{a  - da  + {U-  y/u{U-d))b}. 

Thus,  we  have  Cases  C4-2-1  (x(0)  G {da  + {U-  \/U{U  - d))b,  d{a  + 6)))  and  C4-2-2  (x(0)  G 
[</(«  — \-fjzj),  da-\-{U  — \JU {U  — <i))6]).  For  Case  C4-2-1,  we  will  obtain  a Type-3  Structure, 
and  for  Case  C4-2-2,  we  will  obtain  a Type-2  Structure. 

Case  C4-2-1:  x(0)  G {da  + {U  - y/U{U  - d))b , d{a  + b )). 

According  to  the  quadratic  function  in  Equation  (4.173),  we  have 

JgJ2{S)>  0 if  and  only  if  5 G (52(a;(0)),  S'2).  (4.176) 

Since  52(x(0))  < 0 and  the  range  in  (4.162)  is  [0,a),  we  conclude  from  Equation  (4.176), 

^J2(S)>0,  for  5 G [0,5"). 

Consequently,  5 = 0 solves  (4.162).  The  optimal  production  control,  the  optimal  trajectory, 
and  the  costate  function  are  therefore  the  same  as  in  Case  C4-1. 

Case  C4-2-2:  x(0)  G [d{a  - |^),  da  + {U  - y/U{U  - d))b}. 

According  to  Appendices  C.ll  and  C.12,  we  have 

S2(*(0))  < a - < a,  for  x(0)  G {d{a  - + {U  - yJu{U  - d))b). 

Due  to  Equation  (4.175),  5 = ^(xfO))  solves  (4.162).  Thus,  we  have  a Type-2  Structure 
with  a switching  point  at  to  = a — 52(x(0)).  The  optimal  control  policy  satisfies 


u*(t)  = < 


0,  if  t G [0,a  — 52(x(0))), 

U,  if  t G [a  - 52(a:(0)),  a). 


(4.177) 


For  the  trajectory  x*(t),  we  have 
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x(0)  - dt , if  t £ [0,  a - S2(x(0))], 

**(*)  = x(0)  -dt  + U{t  - (a  - S2(x(0)))),  if  t e [a  - S2(x( 0)),  a],  (4-178) 

x(0)  — dt  + U 52(x(0)),  if  t e [a,a  + b]. 

For  the  function  p*(t ),  we  have  a root  at  the  switching  point,  i.e.,  p*(a  - 52(x(0)))  = 0. 
Together  with  Equation  (4.113),  we  derive 


P*(t)  = < 


n 


a— 52(x(0)) 


2 x*(r)dr,  if  t 6 [0,  a - S2(x(0))], 


fa-s2(x(0))  -2 x*(r)dr,  if  t 6 [a  - S2(x( 0)),  a]. 
For  t e [0,a  - S2(x(0))],  this  yields 


ra-S2(x(  0))  r 

(t)  — J 2 x*(r)dr  = J 


P 


CL  — $2  (x(0)) 


2(x(0)  — dr)dr 


= 2x(0 )(a  - 52(x(0))  - t)  - d((a  - S2(x( 0)))2  - r). 


For  t G [a  — 52(x(0)),  a],  this  yields 
P*(t ) 

= [ —2  x*(r)dr 

J a— S2(a:(0)) 

= -2  /*  {x(0)  — e?(a  — S2(x(0))) 

Xa-S2(a;(0)) 

+(tf  - d)(r  - (a  - 52(x(0)))}dr 


■f 

J a— 


= -2  1 (x(0)  — U(a  — 52(x(0)))  + (17  — d)r}dr 

-S2(x(0)) 

= — 2(x(0)  - tf(a  - S2(x(0))))(f  - (a  - 52(x(0))))  - (U  - d)(t2  - (a  - S2(x(0)))2). 

In  summary,  we  have 

f 

2x(0 )(a  - 52(x(0))  - 1)  - d((a  - 52(x(0)))2  - t2), 
if  t G [0,a  - 52(x(0))]; 

*>*(*)  = — 2(x(0)  - U(a  - S2(x(0))))(t  -(a-  S2(x(0))))  (4-179) 

-{U  - d)(t2  - (a  - S2(x(0)))2), 
if  t £ [a  - S2(x(0)),a]. 
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Finally,  we  verify  the  boundary  condition  (4.115).  With  x*(a)  = x(0)  -da  + U S2(x( 0)), 

P*(a) 

= -2(*(0)  - U(a  - 52(x(0))))(a  - (a  - S2(x( 0))))  - (U  - d)(a2  - (a  - S2(x(0)))2) 

= 2 ( x ( 0 ) - U (a  - 52(®(0))))52(*(0))  - (U  - d)(2aS2(*(0))  - S2(x(0))2) 

= ~(U  + d)(S2(x( 0)))2  - 2(*(0)  - da)S2{x{ 0)), 

and 

d j*(x(a\  _ db)  = (x(a)~db)2  _ (^(0)  ~ d(a  + b ) + E52(z(0)))2 

dx(a)  U ^ 7 ' C7-d  £7-d 

we  have  according  to  Equations  (4.173)  and  (4.174) 

2 x*(a)b  - db 2 - p*(a)  + ^ J}(x(a)  - db) 

= 2{z(0)  - da  + US2(x(0))}b  - db 2 + (£7  + d)(S2(x(0)))2  + 2(x(0)  - da)S2{x{0)) 

(x(0)  — d(a  + 6)  + £7  52(a;(0)))2 
U -d 

= 2x(0)b  — 2dab  + 2U  S2(x(0))b  — db2 

+(U  + d)(S2(x(0)))2  + 2*(0)52(*(0))  - 2daS2(x{0)) 

(*(0)  - d(o  + 6))2  + 2(®(0)  - d(a  + b))US2(x(0))  + U2(S2(x(0)))2 

U-d 

= 2(U  ~ d)x(0)b  - 2(77  - d)dab  + 277(7/  - d)S2(x(0))b  - (U  - d)db 2 

+(U 2 - d2)(S2(x( 0)))2  + 2(U  - d)x( 0)S2(x(0))  - 2 (U  - d)daS2(x( 0)) 

-(*(0)  - d(a  + 6))2  - 2(*(0)  - d(a  + b))US2(x( 0))  - U2(S2(x( 0)))2} 

= ^-^{2(77  - d)®(0)6  - 2{U  - d)dab  + 2U2S2(x(0))b  - (77  - d)db2 
-d2(S2{x( 0)))2  - 2dx(0)52(x(0))  + 2d2aS2(x(0)) 

— (*(0)  — d(a  + 6))2} 

= 0)))2  + 2(U2b  - d(x( 0)  - da))52(x(0)) 

-(x(0)  - d(a  + 6))2  + 2(7/  - d)6®(0)  - (U  - d)(2dab  + db2)} 

= jj^{-d2(S2(x( 0)))2  + 2 (U2b  - d(*( 0)  - da))S2(x(0)) 


134 


-x2(0)  + 2 d(a  + b)x(O)  - d2(a  + b f + 2 {U  - d)bx(O)  - (U  - d)(2dab  + db2)} 

= jj^{-d\S2(x{ 0)))2  + 2{U2b  - d(x( 0)  - da))S2(x(0)) 

-x2(0)  + 2 (Ub  + da)x(O)  - d2(a2  + 2 ab  + b 2)  - d(U  - d){2ab  + b2)} 

= JfTd{-d2(S2(x( 0)))2  + 2(U2b  - d(x(0)  - da))S2(x(0)) 

-z2(0)  + 2{Ub  + da)x{ 0)  - d2a2  - Ud(2ab  + b2)} 

= 0.  (4.180) 


Thus,  the  boundary  condition  is  verified. 

This  completes  the  analysis  for  the  case  S^fa^O))  > 0 which  corresponds  to  the  situation 
x(0)  G [d{a  - \j^),da  + (U  - ^U{U  - d))b\. 

Case  C5:  adO)  G (O.dla- 

First,  since  x(O)  > 0,  the  initial  control  level  satisfies  w*(0)  = 0 according  to  Equation 
(4.123).  Concerning  the  optimal  control  structure,  we  eliminate  a Type-2  and  a Type-3 
Structure  with  similiar  reasoning  as  for  Case  B5  in  Section  4.3.4.  Therefore,  we  must  have 
a Type-1  Structure  which  has  a singular  interval. 

The  left  endpoint  satisfies  t\  = At  time  ti,  the  trajectory  for  z(0)  G (0 ,d(a-\jf^)) 
meets  the  trajectory  Xp(t).  By  the  Principle  of  Optimality,  these  trajectories  coincide  from 
then  on.  Therefore,  the  right  endpoint  is  f2  = a — 

Thus,  the  optimal  control  has  the  singular  interval  [^,<1  - \jjZd)  and  satisfies 


u*(t)  = < 


0, 

d, 


if*e[o,^), 

if  t e [^,a  - \iTZd), 


(4.181) 


U,  if  t 6 [a- 
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The  optimal  trajectory  x*(t)  satisfies 

‘‘C-f-t), 


x*(t)  = < 


0, 


if  t e [o,^], 

if  t e [^,a-  \ 


(4.182) 


(U-d)(t-(a-  i^)),  if<6[a- 

(C  - d)\jfzd  ~ d(t  - a),  if  t € [a,  a + &]. 


I_4T_  al 
2 (/-<*’ “b 


Concerning  the  function  p*(t),  we  must  havep*(£^)  = 0 and  p*(^p)-p*(f)  = — p*(t)  = 


Jt  d —2 x*(r)d,T.  Therefore,  we  have  for  t £ [0, 

g(0)  g(0) 

p*(t)  = f 2 x*(r)dr  = f 2 - T)dr  = -d( 

Jt  Jt  a 


*(o) 


*(°) 


— T 


>2|  d. 


d( 


x(o) 


-tf. 


The  remaining  interval  [^^,a]  results  in  the  same  expression  for  p*(t)  as  in  Cases  Cl 
and  C2  (see  Equations  (4.141)  and  (4.148)),  since  the  trajectory  x*(t)  for  t £ «+&] 

is  identical  to  the  optimal  trajectories  in  Cases  Cl  and  C2. 

In  summary,  we  have 


/>'!*)  = 


(l-  ‘O  e l^.a  - 1’ 

-(V  - i)(t  - (a  - 


(4.183) 


Since  the  values  x*(a ) and  p*(a)  are  the  same  as  in  Cases  Cl  and  C2,  the  boundary 
condition  is  verified  in  the  same  way  as  in  Cases  Cl  and  C2. 

This  completes  our  discussion  of  the  case  x(0)  £ (0,d(a  — |jnrj))- 
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Characterization  of  the  optimal  solution 

Summarizing  Equations  (4.139),  (4.145),  (4.154),  (4.164),  (4.177),  and  (4.181),  we  have 
27,  if  ®(0)  < —{U  - d)(a  - ^ and  /e[0,a); 


u,  ifx(0)e(-([f-d)(a-i^),0),  and  te[  0,^), 

d,  ifz(0)e(-(t/-d)(a-II^),0),  and  <e[^,a-i^), 

t/,  if  x(0)  € (-(U  - d)(a-  5^3),  0),  and  < € [a  - ij^j,a); 


d,  if  x(0)  = 0,  and  t £ [0,  a — | j^j), 

27,  if  z(0)  = 0,  and  < 6 [o- |j^j,o); 


“2(0  = 


0, 


d, 


U, 


if*(0)€(0 ,rf(a-|tfe)),  and  <€[0,^), 

if  z(0)  G (0,  d(a  - | f^rj)),  and  * 6 [^ia_  2^)1 

if  *(0)  G (0,d(a- i^)),  and  « € [a  - a); 


0,  if  x(0)  € [d(a- 

da+  (U  — \/27 (£7  — d))6),  and  t (E  [0,a  - S2(*(0))), 
tf,  if  *(0)  € [d(a-  1^3), 

da  + (U  — \/U(U  — d))6),  and  t € [a  — ^(^(O)),  a); 


0,  if  ®(0)  > da  + (U  - y/ U{U  - d))b,  and  ie[0,o). 


(4.184) 


We  observe  that  during  the  first  cycle,  we  have  a singular  interval  (Type-1  Structure)  for 
z(0)  G (— ( U - d)(a  — | d(a  - \ jy_ zj)),  a switching  point  (Type-2  Structure)  for  £(0)  6 
[d(a  — ^jj^),da+(U  — y/U(U  — d))b),  and  Type-3  Structures  for  x(0)  < — (U — d)(a— 


da-\- 

(i u - vu(u  - d))b 
d(a  ~ \xf^d) 
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Figure  4.16.  Optimal  Trajectory  for  Problem  P2. 

(maximum  production),  and  for  x(0)  > da  + (U  - \/U(U  - d))b  (no  production).  Figure 
4.16  depicts  some  optimal  trajectories.  For  an  initial  production  surplus  level  £(0)  within 
the  range  [-(U  - d)(a  — |j^),  d(a  - \ the  trajectory  will  reach  the  zero  production 

surplus  level  at  a time  t where  t 6 [0,a  — The  trajectory  will  then  stay  at  the  zero 

production  surplus  level  until  time  t = a - For  t e [a  - the  machine  is 

operated  at  maximum  production  rate  before  maintenance  takes  place  during  the  interval 
[a,a  + b]. 

For  an  initial  production  surplus  level  z(0)  within  the  range  (d(a  - \ jjZd),da  + (U  — 
\/U(U  — d))b ),  the  production  rate  initially  is  zero  and  the  production  surplus  consequently 
decreases.  At  time  to  = a — 52(x(0)),  the  production  rate  switches  from  0 to  U.  All  the 
switching  points  form  a switching  curve  xs{to)  which  is  defined  by  the  equation 

S2(xs(to)  + dt0)  = a - t0,  for  t0  G [a  — Cr,a].  (4.185) 

Using  Equation  (4.174),  we  can  solve  for  xs(to)  and  obtain 

for  t0  e [a- a],  (4.186) 


xs(t0)  = Ub  — y/UbVtr^dy/b  + 2(a  - t0), 
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At  the  endpoints  of  the  interval  [a  — we  have 


xs{a  — = Ub  - VWby/U^dJb  + 2(o  - (a  - ^77^3)) 


2U -d‘ 


2U -d' 


= ub  - VubVTT^dJb  + 


db 


U -d 


= Ub  - \fUbVJT^d\  b 


u 

U -d 


= 0, 


and 


xs(a)  = Ub  — \/Uby/U  - d\Jb+  2 (a  - a) 
= Ub-  VUbVU  - dVb 
= b(U  - \Ju(U  - d)). 


Notice  that  the  switching  curve  xs{t)  for  t £ (a  - \ lies  above  the  trajectory  x^t), 

since  we  have  xs(a  - — x^ (a  - = 0 and 


dt 


xs(t) 


-VUbs/TT^dl- 


2 sjb  + 2 (a  — 1 0) 


(-2) 


> 


y/UWU^d 

y/b  + 2 (a  — to) 

VUby/UUUd 

y/b  ’ 


since  to  < a 


= y/U(U-d) 

> U-d 
d 


for  t £ \a  — 


1 db 


2U  — d 


dt 


*o(0» 


Thus,  the  optimal  trajectory  for  an  initial  production  surplus  level  x(0)  £ (d(a-^jj^),  da+ 
( U — y/U(U  — d))b)  does  not  touch  the  trajectory  Xo(t). 


Since  the  optimal  production  policy  u%(t)  is  determined  by  the  initial  production  surplus 
level  ar(0),  we  can  therefore  express  the  optimal  performance  measure  in  dependence  of  z(0) 
as  well.  The  optimal  performance  measure  J|(a;(0))  satisfies 
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/oa{z(0)  + (U  - d)t}2dt  + /ao+!>{:r(0)  + Ua  - dt}2dt 
+Ji(x(0)  + (U-d)a-db), 


-*(°)  1 db 

f0u-d  {*(0)  + (U-  d)t}2dt  + fJu-d{(U  - d)t}2dt 

+ jSi(U  - d)\j£-d  - dt}2dt  + J*x{-\db), 

for  x(0)e[-(U-d)(a-lj^),0], 


1 db 

tia-‘{(U  - d)t}2dt  + J*{(P  - i) !A  - rfl)2*  + 

for  a:(0)  = 0, 


j;(x( o))  = 


x(0)  1 db 

f0  d {x(0)  - dt}2dt  + f02  u~d{(U  - d)t}2dt 
+ Joi(U  - d)\fa  - dt}2dt  + J{{-\db), 

for  ®(0)  € [0,rf(a-  \ 


(4.187) 


/o«-S2(x(o)){z(0)  - dt}2dt 

+ /o52("(O)){^(0)  - d(a  - 52(*(0)))  + (U-  d)t}2dt 
+ XT +6{^(0)  + US2(x{ 0))  - dt}2dt 
+j?(x( 0)  + US2(x(0))  - d(a  + &)), 

for  ®(0)  € [d(a  - \j^g),da  + (U  - ^U(U  - d))b]. 


Joa+b{x(0)  - dt}2dt  + Jx(x( 0)  - d(a  + &)), 

for  x(0)  >da  + (U-  ^U(U  - d))b. 
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In  Appendix  C.8,  we  verify  the  following  property: 


d^o)J2(xo)  < 0,  for  z(0)  < 0, 

d^o)J2(xo)  = 0,  for  x(0)  = 0, 

^0)^2  (xo)  > o,  for  s(0)  > 0. 

From  Equation  (4.188),  we  conclude  for  arbitrary  initial  surplusses  ^(O)  and  a;2(0) 


(4.188) 


</2*(*l(0))  < ^(*2(0)) 

J2*(.n(0))  < /2 (*2(0)) 
According  to  Equation  (4.187),  we  have 


for  0 < a:i(0)  < z2(0), 
for  ar2(0)  < Xi(0)  < 0. 


(4.189) 


d 


dx(  0) 


J*2{x(Q)) 


1 


U-d 


(*(0 ) + («/-  rf)( 


2*<°)7r^!  + (V  - <t)( 


U-d 


~»(0) 

U-d 

U-d ’ 


))2+  / 

Jo 


-*(») 

U-d 


2(®(0  ) + (U-d)t)dt 


x2(0) 

'U-d' 


for  z(0)  G [~(U  - d)(a  - (4-190) 


Also  note  that  J2(x(0))  has  the  following  properties 

d$rj;ix{  0))  = Iud,  + C2dt  + d$rJ:{xm+{V-d)a-db>>0' 

for  *(0)  < -(U  - d)(a  - (4.191) 

and 


d^J*{xm  = IT^d  > °’  f°r  ^0)  e - <*)(«  - \ cTT^)’0)-  (4-192) 

We  now  show  that  J2(x(0))  is  strictly  convex  for  x(0)  < 0 by  showing  that  the  derivative 
of  J2(x( 0))  is  continuous  for  x(0)  = -{U  - d)(a  - The  continuity  of  j^J2  (z(0)) 

at  that  point  is  established  by  showing  that  the  left- hand-limit  and  the  right-hand-limit 
are  identical.  The  expression  for  the  left-hand-limit,  i.e.,  for  an  initial  production  surplus 
satisfying  z(0)  < -{U  - d)(a  - is  obtained  as  follows, 

^yJ2*(a:(0))  = 2(*(0)  + (U  - d)tdt  + 2(*(0)  + Ua  - dt)dt 
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+ d^0 jJ*i(x(°)  + (u-d)a~db) 

= 2x(0)a  +(U-  d)a 2 + 2x(0)b  + 2Uab  - d((a  + b)2  - a2) 

+ d^JttxW+(U-d)a-db) 

= 2x(0)(a  + b)  + (U-  d)a 2 + 2 Uab  - 2 dab  - db 2 

+^o)'7l*(x(0)  + {u~  d)a  ~ db) ■ (4.193) 

In  order  to  complete  the  analysis  of  the  left-hand-limit,  we  need  to  evaluate  the  expression 
d^ojJi(z(0)  + (U  - d)a  - db).  Notice  that  for  x(0)  = -( U - d)(a  - we  have 

z(0)  + (U  — d)a  — db  = —\db  £ (—(17  — d)(a  — Cr ), 0)  and  thus 

^J*(x(0)+(U-d)a-db)  = for  *(0)  = -(U -d)(a~-^).  (4.194) 

With  Equations  (4.193)  and  (4.194),  we  now  verify  that  0))  is  continuous  at 

x(0)  = -(U-d)(a-l^-d), 

lim  - ^ ./.N^IO)) 

*(0 )—(U~d)(a-^),  x(0 )<-(U-d)(a-^)  dx( 0) 

= —2(17  - d)(a  - —Ma  + b)  + (U  - d)a2  + 2(C7  - d)ab  - db2  - 

z u — a U — cl 

= —2(U  - d)a2  -2 (£1  - d)ab  + d6(o  + &)  + (£!-  d)a2  + 2{U  - d)ab  - db2  - 

U — d 

= -(U-d)a2  + dab-t^f 

(U  - dfa2  - (U  - d)dab  + {-\dbf 
U -d 

(-(tf-rf))V-2alfe  + (- 1^)2 

U -d 

(-(E-d))2(a-^)2 

U -d 

U -d 

= Um  .-r-—- J2*(x(0))  (4.195) 

*(°)— (U-d)(a-^),  *(0 )>-([/-d)(o-II^)  d*(0) 

Therefore,  we  conclude  that  J^adO))  is  strictly  convex  for  ,t(0)  < 0. 

This  concludes  our  analysis  of  Problem  P2-  In  the  next  section,  we  study  the  IV-cycle 


problem. 
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4.5  Optimal  solution  to  the  N- cycle  problem  (N  > 3) 

In  the  previous  section,  we  solved  the  two-cycle  problem  P2.  In  this  section,  we  solve 
the  iV-cycle  problem  based  on  the  solution  of  the  N - 1 cycle  problem  for  N > 3. 

The  time  interval  for  the  iV-cycle  problem  is  [0 ,N(a  + &)).  During  the  interval  [a  + 
6,  N(a  + b)),  the  optimal  control  can  be  determined  based  on  the  solution  of  Problem  P/v-i- 
Further,  (a,  a + b)  is  a maintenance  interval.  Thus,  it  remains  to  determine  the  optimal 
control  during  [0,  a]  in  order  to  solve  the  N-cycle  problem. 

The  cost  during  the  interval  [a,  N(a  + 6)]  is  given  by  the  expression  hi\r(x(a)): 

hN(x(a))  = h1(x(a))  + J^f_1(x(a)  - db). 

The  expression  h^(x(a))  is  referred  to  as  terminal  cost  at  time  t = a and  depends  on  the 
production  surplus  level  at  that  time.  It  reflects  the  penalty  during  the  interval  [ a,N(a  + b )). 
The  function  h^^x^a))  satisfies 

^-^;v(z(a))  = 2 x(a)b  - db 2 + i(*(«)  - db).  (4.196) 

The  control  problem  for  the  IV-cycle  case  can  be  restated  as  follows. 


Problem  P/v:  Determine  an  admissible  control  u*(t),  i.e.,  u*(t)  € [0,  U],  for  t e [0,a],  that 
causes  the  system 


dx(t) 

dt 


= u(t ) — d 


to  follow  an  admissible  trajectory  x*(t)  which  minimizes  the  performance  measure 


ra 

Jn(u)  = / x2(t)dt.  + hpf(x(a)). 
J o 

The  boundary  conditions  are  x(0)  = xq  and  x(a)  free. 


Notice  that  the  only  difference  between  Problems  P1?  P2  and  P/v  (N  > 3)  is  the  terminal 
cost.  We  will  see  that  the  analysis  of  Problem  P/v  is  therefore  similiar  to  the  analysis  of 
Problems  P\  and  P2.  For  example,  the  Hamiltonians  for  Problems  Pi,  P2,  and  P N are 
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independent  of  the  terminal  cost  and  are  therefore  identical.  According  to  Pontryagin’s 
Minimum  Principle,  the  necessary  optimality  conditions  for  Problem  P 'n  are  as  follows: 

= u*(t)-d , (4.197) 

= -2  x*(t),  (4.198) 

= “in  >*2(0  + p*(t)[u(t)  - d]} 

«p)e[  o,c7j 

= x*2(t)  - p*(t)d  + min  {p*(t)u(t)}.  (4.199) 

u(t)e[o,[/] 

The  boundary  condition  for  Problem  P /v  is 

2x*(a)b  - db 2 - p*(a)  + J^(x(a)  - db)  = 0.  (4.200) 

In  addition,  we  have 


dx*(t ) 
dt 

dp*(t) 

dt 

x*2(t)  + p*(t)[u*(t)  - d] 


X*2(t)  + P*(t)[u*(t)  - d]  = c 1,  for  t 6 [0,  <z],  (4.201) 


and 


u*(t) 


0,  if  p*(/)  > 0, 

U,  if  p*(t)  < 0, 

/3,  ifp*(/)  = 0. 


(4.202) 


where  (3  £ [0,  U]  is  to  be  determined. 

According  to  Equation  (4.202),  the  optimal  control  law  u*(t)  is  thus  determined  when- 
ever we  have  p*(t ) ^ 0.  In  the  case  that  roots  exist  for  the  function  p*(t ) within  the  interval 
[0,a],  in  the  same  way  as  for  Problem  P2,  we  further  distinguish  between  singular  intervals , 
switching  times,  and  switching  points.  The  definitions  of  a singular  interval  and  a switching 
point  in  Section  4.4  (see  Equations  (4.18)  and  (4.19))  apply  here  since  we  are  consider- 
ing the  first  cycle.  The  corresponding  results  regarding  the  possible  control  structures  are 
summarized  in  the  following. 
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4.5.1  Singular  Intervals  for  Problem  Pn 

If  there  is  a singular  interval  [ti,t2]  C [0,a],  the  optimal  control  satisfies 


u*(t)  = < 


Pi, 

d, 

U, 


where  Pi  G {0,  U}. 

4.5.2  Switching  Points  for  Problem  Pm 


if  t G [(Mi), 
if  t € [fi?  ^2), 
if  t G [t2,a), 


(4.203) 


If  u*(t.)  is  an  optimal  control  with  a switching  time  t0  G (0 ,a),  necessary  optimality 
conditions  for  the  switching  point  and  the  initial  production  surplus  level  are 


x*(t0)  > 0, 

x(0)  > 0, 


and  it  is  further  necessary  that  u*(t)  satisfies 

0,  if  t G [0,  to ), 

u*(t)  = < 

U , iftG[to,a)- 

4,5.3  Initial  Control  Level  for  Problem  Pm 


The  initial  control  level  u*(0)  satisfies 

«*(0) = < 

4.5.4  Optimal  Solution  for  Problem  Pat 


0, 

d, 

U, 


if  a;(0)  > 0, 
if  a;(0)  = 0, 
if  x(0)  < 0. 


(4.204) 

(4.205) 


(4.206) 


(4.207) 


In  this  section,  we  solve  for  the  optimal  solution  to  Problem  P )y.  Based  on  the  discussion 
above,  the  optimal  control  policy  has  three  possible  structures  as  defined  in  Section  4.4.4. 
In  order  to  facilitate  the  analysis,  we  partition  the  range  of  a;(0)  in  the  following  way: 


Dl: 


x(0)  = 0, 
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D2:  x(0)G(-(C/-d)(a-i^),0), 

D3:  x(0  )<~(U-d)(a-l-1^), 

D3-1:  *(0)  e[-(U-  d)a , -( U - d)(a  - 

D3-2:  x(0)  < -{U  - d)a, 

D4: 

D4-1:  x(0)  > d(a  + b), 

D4-2:  x(0)  € [d(a  - | ^^),d(a  + 6)), 

D5:  *(0)e(0,^-~)). 


The  derivation  of  the  optimal  control  for  Problem  P/v  for  N > 3 is  based  on  the  following 
properties  of  Problem  P/v-i- 


1.  Property  1.  The  first  property  concerns  the  derivative  of  the  performance  measure  for 
a certain  range  of  x(0): 


d T*  /wrm-  *2(°) 

dx(0)JN~l(  (0))  U-d' 


1 db 


for  x(0)  E [~{U  - d)(a  - - ^),0].  (4.208) 


For  N - 1 = 2,  see  Equation  (4.190).  Notice  that  Equation  (4.208)  implies 

d j*  ((TT  ,,  x tTj  ,v  (( U - d)C  - db )2 

- d)C  - db)  = (U  — d){  — — i-) 


U-d 
-((U  - d)C  - db)2 


= -{U-d)2(C- 


db 

U-d 


)2, 


(4.209) 


for  (U-d)C-db  E [-(£f - d)(a - 5^), 0],  or  equivalently,  C E [-(a- 
In  particular,  since  G [-(a  - f cnr^),  we  have 

- d)C  - db)  = -(^d&)2,  for  C = (4.210) 

2.  Property  2.  The  function  </jv_i(z  (0))  is  strictly  convex  for  x(0)  < 0.  For  N — 1 = 2, 
see  Equations  (4.191),  (4.192),  and  (4.195). 
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3.  Property  3. 


l$®J*N-i(xo)  < o,  for  x(0)  < 0, 


d$o)JN-i(xo)  = 0,  for  *(0)  = 0, 


(4.211) 


d$o)JN-i(xo)  > 0,  for  *(0)  > 0. 


Equivalently,  we  can  write 


</W*i(0))  < ^Jv-rC^CO))  if  0 < *i(0)  < *2(0), 


(4.212) 


^-i(*i(0))  < ^_!(*2(0))  if  *2(0)  < an(0)  < 0. 


We  have  shown  that  the  three  properties  above  are  valid  for  N = 2.  Assuming  that  they 
are  true  for  arbitrary  N - 1 (N  > 3),  we  will  show  by  induction  in  Section  4.5.4  that  these 
properties  are  valid  for  general  N. 

Case  Dl:  xfO)  = 0 

From  Equation  (4.207),  we  obtain  for  the  initial  control  level  u*(0)  = d for  x(0)  = 0. 
With  the  same  considerations  as  in  Section  4.4.4,  we  conclude  that  the  optimal  production 
control  has  a Type-1  Structure  with  a singular  interval  [0,a  - C]  where  C € [0,a).  The 
performance  measure  can  be  expressed  in  terms  of  C and  satisfies 


+J*N-i({U  - d)C  - db). 


(4.213) 


We  seek  to  determine  the  value  of  C G [0,  a)  which  solves 


min{/jv(C')  : Ce[0,a)}. 


(4.214) 


We  differentiate  Jat(C)  by  using  Leipniz’  rule  and  obtain 


{{U  - d)Cf  + j 2(17  - d){(U  - d)C  - dt)dt  + - d)C  - db) 


(U  - df{C 2 + 2 bC  - - —b2}  + - d)C  - db). 


(4.215) 
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For  C = we  have  -^J%_ j((I7  — d)C  — db ) = — (|d6)2  according  to  Equation  (4.210), 

and  thus 


d , — , . . db  . o , 1 db 

—JN(C)  ~ ( U-d)2{{ -P  + 26 

dC  ‘ y ’ y ’ XK2U-d>  2U-d  U-d 


d b2)-(\dbf 


2 f„,  1 db 


d 


= U-d 


i2} 


= 0, 


for  C = 1-db 


2 U-d 


(4.216) 


The  second  derivative  of  Jjy(C)  satisfies 


^Jn(C)  = (U-  df{2C  + 26}  + - d)C  - db).  (4.217) 


Since  Jjv-i(*(0))  is  strictly  convex  for  x(0)  < 0,  we  have 

- d)C  - db)  > 0,  for  c < 

Thus,  we  conclude  with  Equation  (4.217)  that  JN(C)  is  strictly  convex  for  C < and 
thus,  C = solves  (4.214).  Therefore,  for  the  first  cycle,  we  have  the  same  optimal 

control,  optimal  trajectory,  and  costate  function  as  in  Case  Cl  (see  Equations  (4.139), 
(4.140),  and  (4.141)). 

Since  we  have  3^0) ^-l^0))  = for  *(0)  € (-(U  - d)(a  - |^),0)  according 

to  Property  1,  we  conclude  = ~ %d-d  • Since  further  the  expressions  for 

x*(a)  and  p*(a ) are  the  same  as  in  Case  Cl,  the  boundary  condition  (4.200)  is  verified  in 
the  same  way  as  in  Equation  (4.142). 

Optimality  Condition 

If  x$(t)  and  x\ (t)  are  two  optimal  trajectories  for  Problem  P/v  with  initial  production 
surplusses  x}(0)  and  x^O),  respectively,  then,  the  trajectories  will  not  cross  each  other,  i.e., 


if  Xi(t')  < x\ (/')  for  some  t'  £ [0,a],  then  x}(t)  < x*2{t)  for  all  t £ [0,o],  (4.218) 

and  in  particular  for  t'  = 0,  we  have 


if  x}(0)  < xj(0) 


then 


x\(t)  < x*2 (t),  for  t £ [0,  a]. 


(4.219) 
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Case  D2:  x(0)  G (-(U  - d)(a  - i^j),  0) 

From  Equation  (4.207),  we  have  it*(0)  = U.  Since  the  initial  production  surplus  is 
negative,  we  can  exclude  a Type-2  Structure  (switching  point).  Since  the  optimal  trajectory 
x*(t)  can  not  cross  the  trajectory  x^t),  we  conclude  that  it  has  a Type-1  Structure  with 
singular  interval  [ , a - |^j]-  The  optimal  control,  the  optimal  trajectory,  and  the 

costate  function  are  thus  governed  by  Equations  (4.145),  (4.146),  and  (4.148). 

Case  D3:  afO)  < -(U  - d)(a  - 

First,  since  x(0)  < 0,  the  initial  control  level  satisfies  u*(0)  = U according  to  Equation 
(4.207).  Therefore,  a Type-2  Structure  can  be  excluded,  since  it  requires  u*( 0)  = 0.  Thus, 
the  optimal  control  has  either  a Type-1  Structure  or  a Type-3  Structure.  That  is,  the 
optimal  control  either  produces  at  maximum  capacity  or  it  has  a singular  interval.  We 
further  partition  the  initial  surplus  into  two  smaller  regions,  x(0)  £ [-(U  - d)a,-(U  - 
d)(a  — (Case  D3-1)  and  a;(0)  < —{U  — d)a  (Case  D3-2),  and  show  that  the  optimal 

control  has  a Type-3  Structure. 

Case  D3-1:  x(0)  € [-(£7  - d)a, -(U  - d)(a  - ±^)]. 

We  either  produce  at  maximum  capacity  (Type-3  Structure)  or  we  have  a singular  interval 
[t\,t2]  (Type-1  Structure).  As  in  Case  C3-1,  we  can  represent  both  control  structures  by 


u(t) 


U, 

d, 

U, 


where  C G [0,a — 

The  performance  measure  satisfies 


Jn(C ) = / 

Jo 


— x(0) 

U-d 


if 

if  t G [a  - C,  a), 


(4.220) 


((U  - d)tfdt  + 
+Jn-\{{U  - d)C  - db). 


rC  rb 

/ ((u-d)t)2dt+  / (( u-d)c-dty 

Jo  Jo 


dt 


(4.221) 
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We  seek  to  determine  the  value  of  C which  solves 

min{J;v(C):  Ce[0,a-^]}. 


(4.222) 


Since  the  first  integral  of  the  right-hand-side  in  Equation  (4.221)  is  independent  of  C,  the 
derivative  ~$jJn(C)  is  identical  to  the  derivative  in  Case  Dl.  Thus,  J/v(C)  is  strictly  convex 
for  C < and  has  a unique  minimum  at  C — Since  we  have 


a — 


-a;(0)  1 db 


U-d~  2U-d’ 


to  *(0)  <-(C -«()(«. -EiE), 


we  conclude  that  C = a - solves  (4.222).  Equation  (4.220)  with  C = a - leads 
to 

u*(t)  = U,  for  t £ [0,  a).  (4.223) 


Case  D3-2:  *(0)  < -(U  - d)a. 

Due  to  x(0)  < 0,  we  have  u*(0)  = U.  Above,  we  have  already  excluded  a Type-2  Structure. 
Here,  we  can  exclude  a Type-1  Structure  with  a singular  interval  as  well,  since  x(0)  < 
—(U  — d)a  implies  that  no  trajectory  can  reach  the  zero  production  surplus  level  for  t € [0,  a]. 
Thus,  we  have  a Type-3  Structure  and  produce  at  maximum  capacity  (see  Equation  (4.223)). 

Thus,  for  the  first  cycle,  we  have  the  same  control  as  in  Case  C3.  The  optimal  control  and 
the  optimal  trajectory  are  therefore  governed  according  to  Equations  (4.154)  and  (4.155), 
respectively.  With  the  boundary  condition,  we  have  for  the  costate  function 


p*(t)  = 2(a  + b - f)z(O)  + (U  - d)(a2  + 2 ab  - f2)  - db 2 + - db).  (4.224) 

cix  y cl  j 

This  completes  the  analysis  of  Case  D3  (z(0)  < -(U  - d)(a  - |^ij)). 

Case  D4:  adO)  > d( a - 

Since  the  optimal  trajectory  can  not  cross  the  trajectory  x^t)  according  to  Property 
(4.219)  (which  satisfies  Xg(t)  > 0 for  t 6 (a  — jj^j?a)),  we  can  exclude  a singular  interval 
(Type-1  Structure)  from  consideration.  The  production  control  for  the  remaining  structures 
(Type-2  and  Type-3)  can  be  combined  as  in  Equation  (4.160).  It  results  in  the  following 
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performance  measure  (see  Equation  (4.161)), 


Jn(S ) = r (x(0 )-dt)2dt  + r(x(0)- d{a- S)  + (U  - d)t)2dt 
Jo  Jo 

rb 

+ / (x(0)-da  + US  -dt)2dt  + Jh^x^  + US  -d(a  + b)).  (4.225) 

•J  0 


We  seek  to  determine  the  solution  of  value  of  S which  solves 


min{JAr(5')  : S £ [0,a)}. 


(4.226) 


Its  derivative  is  (see  Equation  (4.163)) 


±JN(S)  = U(U  + d){S2  + 2^11  *°  + U.S  _ JL(j(,  + 1 ) _ x(0))} 


d 


+ j^J*N-i(x(0  ) + US-d(a  + b)). 


(4.227) 


At  this  point,  we  further  distinguish  the  cases  z(0)  > d{a  + b)  (Case  D4-1)  and  z(0)  £ 
[d(a  - \1jz2),  d(a  + b ))  (Case  D4-2). 

Case  D4-1:  .t(0)  > d(a  + b). 

According  to  Property  3,  we  have  0)  + US  - d(a  + 6))  > 0 for  S £ [0,a)  and 

*(0)  > d(a  + b).  Since  the  first  term  of  the  right-hand-side  of  Equation  (4.227)  is  strictly 
positive,  we  have  ^Jn(S)  > 0 for  S £ [0,a)  and  z(0)  > d(a  + b).  This  implies  that 
the  optimal  control  has  a Type-3  Structure  where  the  machine  is  shut  down.  That  is,  the 
optimal  production  control  satisfies  (see  Equation  (4.164)) 


u*(t)  — 0,  forfe[0,a).  (4.228) 

The  optimal  trajectory  satisfies  (see  Equation  (4.165)) 


x*(t) x(0)  - dt,  for  t £ [0,a  + 6),  (4.229) 

and  the  costate  function  is  governed  by  (see  Equation  (4.166)) 

p*(t.)  = 2x(0)(a  + b~t)-d((a  + b)2 -t2)+ -J^rN_1(x(0)-d(a  + b)). {4.230) 
This  completes  the  analysis  of  the  case  a;(0)  > d(a  -f  b). 


151 


Case  D4-2:  a:(0)  G [d(a  - b)). 

In  this  case,  we  also  have  Equation  (4.227).  We  first  consider  the  range  of  x*(a  + b)  — 
x(0)  + US*  — d(a  b)  where  S*  solves  (4.226).  From  Cases  D1  and  D4-1,  we  have 

x*(a  + b)  — -^d6,  for  a:(0)  = 0, 

x*(a  + b ) = 0,  for  a:(0)  = d(a  + b). 


Since  the  optimal  trajectories  will  not  cross  each  other  (see  the  Optimality  Condition 
(4.219)),  we  conclude 


x*(a  + b)  G [—  -db,  0],  for  x(0)  G [0,  d(a  + b)]. 


(4.231) 


Since  we  have  -\db  > -{U  — d)(a  - we  have  further 


**(“  + »)  6 [-|d6,0]  C [-(P  - <0(o  - \ j^C),0]. 


(4.232) 


According  to  Property  1,  we  have 
d 


dx{  0) 


</jv-i(a;(0)  + US*  — d(a  + b))  = 


x*2(a  + 6) 

U-d  ’ 

for  x*(a  + 6)  G ( — 0). 


Therefore,  Equation  (4.227)  can  be  written  as 


= P(P  H-  *)<  + 2d°)  - * + W5  - ^(<o  + |)  - x(0))} 


+ 


t/ 


U -d 


(x(0)  + US  -d(a  + b))2. 


(4.233) 


This  expression  is  identical  to  the  derivative  in  Case  C4-2  (see  Equation  (4.172)).  Therefore, 
the  remaining  analysis  is  identical  to  Case  C4-2. 

Case  D5:  x(0)  G (0,d(a- 

With  the  same  reasoning  as  in  Case  C5,  we  conclude  that  we  must  have  a singular 
interval  (Type-1  Structure)  (otherwise,  the  optimal  trajectory  for  an  initial  production 
surplus  x(0)  G (0,  d{a  — would  cross  the  optimal  trajectory  for  a;(0)  = d(a—  ^j^rj)). 
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The  left  endpoint  t\  of  the  singular  interval  satisfies  t\  = Since  this  trajectory  now 
meets  the  optimal  trajectory  x^t),  it  coincides  with  (t)  from  then  on  and  the  right 
endpoint  of  the  singular  interval  is  d(a  - Thus,  we  have  the  same  results  as  in  Case 

C5  (see  Equations  (4.181),  (4.182),  and  (4.183)). 

Characterization  of  the  optimal  solution 

Comparing  the  results  for  Problems  P2  and  Pn,  we  obtain 

u*N(t)  = u^t),  for  t £ [0,  a],  (4.234) 

For  the  performance  measure,  we  have 
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/0°{x(0)  + (U  - d)t}2dt  + /aa+!,{z(0)  +Ua-  dt}2dt 
+J*N_1(x(0)  + (U-d)a-db), 
for  x(0 )<-(U-d)(a-\T£3), 

~*(°)  1 db 

foU~d  {*(0)  + (U-  d)t}2dt  + f02  v~d{(U  - d)t}2dt 

+ foi(U  - *)£  A “ dt}2dt  + rN_x{-\db), 

for  x(0)E  [-{V  -d\a-\fy),  0], 


1 db 

So  U~d{(U  - d)t}2dt  + /06{(C/  - d)f  A - dt}2dt  + J*N_x{-\db), 

for  x(0)  = 0, 


^(*(0))  = 


1 db 

{x(0)  - dt}2dt  + f02  u~d{(U  - d)t}2dt 

+ fob{(U  ~ <0|  A - dt}2dt  + Jh-A-ldb), 

for  *(0)  € [0,d(a-  £A)]> 


rS2Wo)){I(o)  - dtfdt 

+ /oS2(l(O)){^(0)  - d(a  - S2(x(0)))  + (U-  d)t}2dt 
+ f:+b{x(0)  + US2(x(0))-dt}2dt 
+J*n-i(x( 0)  + US2(x{ 0))  - d(a  + b )), 

for  *(0)  E [d(a  - da  + (U  - y/U(U  - d))b], 


/oa+6{x(0)  - dt}2dt  + Jjy_1(a:(0)  - d{a  + &)), 


for  x(0)  > da + (U-  y/U(U  - d))b. 


(4.235) 
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In  order  to  complete  our  analysis,  we  need  to  show  that  the  properties  for  Problem  P/v-i 
which  we  listed  at  the  beginning  of  this  section  also  hold  for  Problem  P/v. 

1.  Property  1.  According  to  Equations  (4.235),  (4.187),  and  (4.190),  we  have 

m™*0”  “ for  «°> 6 - ■')(“  - 1- 

2.  Property  2.  The  function  J^{x( 0))  is  strictly  convex  for  x(0)  < 0.  Using  the  strict 
convexity  of  «Uv-i(z(0))  for  x(0)  < 0,  this  property  is  shown  in  the  same  way  as  it 
has  been  shown  for  Problem  P 2. 

3.  Property  3.  From  the  Equations  (4.187)  and  (4.235),  we  see 

(4.236) 

For  x(0)  < -( U — d)(a  - \jpzd),  we  have  (see  Equation  (C.17)) 

^qTJn(x(0))  = £ 2(x(0)+(U -d)t)dt  + £+b  2(x(0)+Ua-dt)dt 

+ dx{ ^ J^-i(x(°)  + (U  ~ d)a  ~ db ) 

= ( U - d){2  jj£j(a  + b)  + (a  + 6)2  - jj—  ^b2} 

+ dx( o)^iV-l(x(0)  + (U  ~ d)a  ~ db)- 
According  to  Equation  (C.40),  we  have 

(U-d){2p^(a+b)+(a+b)2- ££-/}  < 0,  for  x(0)  < -(U -d)(a£-£^), 

and  since  x(0)  + (U  - d)a  - db  < -\db  < 0 for  x(0)  < -( U - d)(a  - we  have 

according  to  Equation  (4.211) 

^7^Tat-i(x(0)  + (U  - d)a  - db)  < 0,  for  x(0)  < -(U  - d)(a  - \ 

In  conclusion,  we  have 

< 0,  fop  x(0)  < -(£/  - d)(a  - i^). 
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For  x(0)  £ [d(a  - ^j^Zd),da  + (U  - \/U(U  — d))b]  and  x(0)  > da  + (U  - \/U{U  — d))b, 
the  result 

™*(x(0))>0,  forx(0  )>i(a-\-^-d 

is  obtained  in  a similiar  way  as  for  Problem  P2  (see  Appendix  C.8). 

Thus,  we  have 

3^oJ</jv(xo)  < 0,  for  x(0)  < 0, 

d^o}Jjv(xo)  = 0,  for  x(0)  = 0,  (4.237) 

dJ@jJN(xo)  > 0,  for  x(0)  > 0. 

This  concludes  the  analysis  of  Problem  P jy. 

4.6  Transient  State  and  Steady  State 

In  this  section,  we  analyze  the  transient  state  and  the  steady  state  of  the  system.  First 
of  all,  we  consider  that  the  number  of  cycles  N is  sufficiently  large,  so  that  the  analysis  is 
not  particularly  dependent  on  the  terminal  conditions. 

Figure  4.17  illustrates  some  trajectories  with  different  initial  surpluses.  We  observe  that 
for  x(0)  £ [— (U— d)(a— d(a— all  trajectories  reach  the  point  x*(a-\-b)  = — ^db 
at  the  end  of  the  first  cycle  and  will  follow  an  identical  pattern  for  the  remaining  up-down 
cycles  from  then  on.  This  production  surplus  pattern  is  referred  to  as  the  steady  state 
trajectory.  In  addition,  for  x(0)  ^ [-(U  - d)(a  - \ jjZd)-,d(a  - \xfzj)],  the  trajectory 
contracts  toward  the  zero  production  surplus  level  and  will  reach  x*(n(a  + b))  — —\db  for 
the  first  time  for  some  finite  n (provided  that  either  x(0)  > -\db  or  ( U - d)a  > db  holds). 
Then,  it  will  follow  the  steady  state  trajectory. 

We  introduce  the  following  definition  for  steady  state:  A system  is  said  to  be  in  steady 
state  after  n cycles  if  the  surplus  trajectory  satisfies  x*(n(a  + 6))  = —\db  for  the  first  time. 

Therefore,  the  system  is  in  transient  state  for  t £ [0,  n(a  + b )]  and  in  steady  state  for 
t > n(a  + b)  (n  < N).  Consequently,  it  would  be  of  interest  to  determine  n in  terms  of  the 
initial  production  surplus. 
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Steady  State  Trajectory 


2/+J>  '2^1^  3o  -f  : 2b  X\+  36 


\ 


Figure  4.17.  Transition  and  Steady  State 

The  transition  from  an  initial  surplus  state  to  the  steady  state  can  be  described  by  the 
use  of  contraction  mappings.  Let’s  consider  the  first  up-down  cycle.  Starting  from  the 
initial  surplus  i(0),  the  optimal  control  policy  will  bring  the  production  surplus  to  a level 
x*(a  + b ).  Let  X(x(0))  be  the  mapping  function, 


X(x(0))  = x*(a  + b). 
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Since  the  optimal  control  policy  is  known,  the  mapping  function  is  well  defined  and  is  given 

by 


x(0)  + (17  — d)a  — db,  for 
-\db,  for 


x(0)<-(C/-d)(a-!^), 


X(z(0)) 


< 

z(0)  + US2{x(  0)) 


-d(a  + b ), 


x(0)  — d(a  + 6), 


^(a  2 lf—d )!’ 

for  x(0)  G (d(a- 

da  + (U  - sJU(U  - d))b), 
for  z(0)  > da  + (£7  - y/U(U  - d))6. 


(4.238) 


Subsequently,  during  the  second  cycle,  the  optimal  control  maps  the  production  level  x*(a  + 
b)  = X(x(0))  to  x*{2 (a  + b ))  = X2(a:(0)).  After  n cycles,  we  have 


x*(n(a  + 6))  = Xn0r(O)). 


Consequently,  the  definition  of  steady  state  can  be  restated  as  follows:  A system  is  said  to 
have  reached  the  steady  state  after  n cycles  if 


^ -i db, 

and  Xm(a:(0))  = -\db,  for  n < m < N. 


(4.239) 


According  to  the  definition  (4.239)  and  Equation  (4.238),  we  now  determine  n for  a given 
x(0).  Since  we  must  have  x*(n(a  + b ))  = -\db  according  to  definition  (4.239),  x(0)  and  n 
must  satisfy 


x(0)  + n((U  - d)a  - db)  > 
a:(0)  — n(d(a  + b ))  < 


~\ih’ 


or  equivalently, 


z(0)  > ~n((U  — d)a  - db)  - 

*(0)  < n(d(a  + b))  - ^db. 


if  z(0)  < ~^db, 
if  x(0)  > — ^ db . 

if  x(0)  < — ^db, 
if  z(0)  > ~7^db. 


(4.240) 

(4.241) 
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Thus,  we  have 


n - 


' d(a+b)  I ’ 

r-(x(°)+id6)n 
I (U-d)a-db  I: 


for  x(0)  > - \db , 

for  x(0)  < -\db  and  ( U - d)a  - db  > 0. 


(4.242) 


The  system  does  not  reach  the  steady  state  for  x(0)  < -\db,  if  (U  - d)a  - db  = 0.  That  is, 
the  system  has  only  marginal  capacity  and  the  surplus  level  can  never  reach  -\db. 


4.7  Summary 

In  this  chapter,  we  study  a production  flow  control  problem  with  periodic  maintenance. 
The  system  consists  of  a single  machine  and  produces  one  product-type.  The  demand  rate 
is  constant.  The  objective  is  to  trace  the  demand  rate  as  closely  as  possible.  Due  to  the 
maintenance  disruption,  the  production  deviates  from  the  demand.  An  optimal  control 
model  is  formulated  to  track  the  demand  as  closely  as  possible. 

We  have  used  Pontryagin’s  Minimum  Principle  to  solve  the  one-cycle  problem  P\  to 
optimality.  We  have  discovered  that  the  structure  of  the  optimal  control  is  dependent 
on  the  initial  production  surplus  level  x(0).  We  have  seen  that  there  are  three  possible 
control  structures  which  need  to  be  considered  for  optimality.  A Type-1  Structure  includes 
a singular  interval,  a Type-2  Structure  includes  a switching  point,  and  a Type-3  Structure 
has  neither  a singular  interval  nor  a switching  point. 


The  optimal  control  structure  for  the  various  ranges  of  x(0)  is  largely  governed  by  the 
optimal  trajectory  Xq(/)  for  x(0)  = 0.  We  have  frequently  used  the  Principle  of  Optimality 
in  conjunction  with  the  trajectory  ^(f)  to  identify  the  optimal  production  control  structure. 

Next,  we  have  used  these  techniques  to  solve  the  two-cycle  and  the  IV-cycle  Problem. 
As  already  discovered  for  the  one-cycle  problem,  the  solution  structure  is  different  from  a 
hedging  point  policy.  With  N > 2 cycles,  the  solution  now  has  a symmetric  trajectory  x^{t) 
during  the  first  maintenance  interval  (a,a-f-6).  The  optimal  control  during  the  first  cycle  is 
the  same  as  long  as  we  have  at  least  two  cycles.  The  trajectory  crosses  the  zero  surplus  level 
in  the  middle  of  the  maintenance  interval.  We  identified  solutions  both  for  the  transient 
state  and  for  the  steady  state.  For  a sufficiently  large  control  horizon,  starting  from  the 
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initial  surplus,  the  trajectory  contracts  toward  the  zero  surplus  level  and  eventually  reaches 
the  steady  state.  In  the  steady  state,  the  system  follows  an  identical  trajectory  pattern  for 
each  of  the  remaining  operation-maintenance  cycles. 

The  study  of  this  problem  can  be  considered  as  a starting  point  for  a new  area  of  research. 
Currently,  an  extension  of  the  model  to  a two-machine  tandem  line  is  being  studied  where 
the  machines  have  the  same  maximum  production  capacity.  Different  machine  capacities 
constitute  a further  extension.  Systems  with  flexible  machines  would  be  another  setting 
to  apply  Pontryagin  s Minimum  Principle  as  well  as  the  case  where  the  demand  rate  is  a 
piecewise  constant  function. 


APPENDIX  A 


PROOF  OF  LEMMA  1 

Proof.  For  parts  (a)  and  (6),  we  define  ne,  nt , Ai,  and  A2  as  follows:  Let  ne  be  the 
number  of  jobs  with  Sj  < e and  nt  be  the  number  of  jobs  with  Cj  > d.  Further,  define  Aj 
and  A 2 by 

A!  = min  (min  {e  - Sj\Sj  < e},min{d  - Sj\S:  < d}) 

J i 

A2  = min(min{Cj  - e\ Cj  > e},min{Cj  - d\Cj  > d}) 

J j 

(a)  For  the  decision  variable  case,  let  a be  a schedule  which  satisfies  Cj  ^ d and  Cj  ^ e 
for  all  j. 

If  (ne  — l)a  > ( nt)(3  ((ne  — l)o  < ( nt)(3 ),  shift  the  due  window  [e,d]  to  the  left 
(right)  by  Ai  (A2)  units.  This  will  decrease  (increase)  the  earliness  of  ne  — 1 jobs 
and  increase  (decrease)  the  tardiness  of  nt  jobs  and  results  in  a schedule  satisfying 
either  Cj  = e or  Cj  = d for  some  job  Jy  The  objective  value  of  the  new  schedule  is 
either  maintained  (in  case  of  (ne-  l)a  = ( nt)f3 ) or  improved  by  Ai((ne-  1 )a-(nt)(3) 
(A2((nt)P-  (ne  - l)o)).  Thus,  an  optimal  schedule  exists  satisfying  either  Cj  = e 
or  Cj  = d for  some  j.  Please  note  that  shifting  the  due  window  to  the  left  is  feasible 
(e  > 0)  due  to  the  definition  of  Aj  and  5'[i]  > 0. 

(b)  For  the  parameter  case,  assume  that  an  optimal  schedule  a satisfies  neither  (i),  nor 
(ii),  nor  (iii).  In  this  case,  we  construct  another  schedule  by  shifting  the  jobs: 

If  (ne  — l)a  > (nt)/3  ((ne  — l)a  < ( nt)/3 ),  shift  all  the  jobs  to  the  right  (left)  by  Ai 
(min(A2,  5[j]))  units.  This  will  decrease  (increase)  the  earliness  of  ne  - 1 jobs  and 
increase  (decrease)  the  tardiness  of  nt  jobs  and  results  in  a schedule  satisfying  either 

Cj  = e or  Cj  = d for  some  job  Jj  or  5^]  = 0.  The  objective  value  of  the  new  schedule  is 
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either  maintained  (in  case  of  (ne  — l)a  = ( nt)(3 ) or  improved  by  Ai((ne  — l)a  — ( nt)/3 ) 
(min(A2,  5'[1])((ra<)/3  - ( ne  - l)a)).  Thus,  an  optimal  schedule  exists  satisfying  either 
Cj  = e or  Cj  = d for  some  j or  5[j]  = 0.  Please  note  that  shifting  the  due  window  to 
the  left  is  also  feasible  in  this  case. 

(c)  (i)  Let  a be  an  optimal  schedule  with  Cj  = e for  some  job  Jj.  By  Property  (CDW)2, 

we  know  that  the  E-jobs  are  sequenced  in  LPT  order. 

Assume  that  there  are  two  adjacent  jobs  Jj  and  Jk  ( Cj  = Sk)  with  p3  > pk  which 
both  start  after  e,  i.e.,  Sj  > e.  Construct  schedule  o'  by  interchanging  jobs  Jj 
and  Jk.  We  thus  have  C'j  — Ck  and  Ck  = Cj  — (p3  — pk)  and  therefore 

cost(a')  - cost(a ) = /3((C'k  - d)+ ) - (Cj  - d)+) 

= fi((Cj  - (Pj  - Pk)  - d)+)  - (Cj  - d)+)  < 0 

since  pj  - pk  > 0.  We  repeat  this  process  until  all  jobs  J/  with  5;  > e are 
sequenced  in  SPT  order.  Since  the  cost  does  not  increase,  there  is  an  optimal 
schedule  where  all  jobs  //  with  5;  > e are  sequenced  in  SPT  order.  Since  the 
E-jobs  are  sequenced  in  LPT  order,  we  have  a V-shape  around  e. 

(ii)  The  proof  in  case  of  Cj  = d for  some  j is  symmetric  to  the  case  Cj  — e. 

(iii)  Now  we  consider  the  parameter  case  where  there  is  an  optimal  schedule  a with 
at  least  one  job  which  is  completed  during  the  due  window.  Let  job  J j be  the  job 
which  satisfies  both  e < Cj  < d and  Sj  < e.  By  the  same  interchange  argument 
as  in  the  proof  of  part  (c)(i),  we  can  sequence  jobs  Jk  with  Sk  > Cj(>  e ) in  SPT 
order  and  jobs  Jk  with  Ck  < Cj(<  d)  in  LPT  order  while  either  maintaining  or 
improving  the  objective  value.  Thus,  in  this  case  (5fl]  = 0 and  e < Cj  < d for 
some  j),  there  is  an  optimal  schedule  where  the  jobs  Jk  with  Sk  < e are  sequenced 
in  LPT  order  and  the  jobs  Jk  with  Sk  > e are  sequenced  in  SPT  order. 


QED 
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Lemma  4.  Proof.  We  first  demonstrate  that  an  optimal  solution  exists  satisfying 

Pj  < Pi  for  all  J0  € W(o)  and  Jt  £ E(o)  U T(o).  (A.l) 

According  to  Lemma  1,  parts  (a),  (c)(i)  and  (c)(ii),  it  is  sufficient  to  consider  the  following 
two  cases.  Schedule  o satisfies  either  Ck  = d for  some  job  J k with  a V-shape  around  d or 
Cfc  — e for  some  job  J k with  a V-shape  around  e. 

1.  Ck  = d for  some  job  J k with  a V-shape  around  d: 

Let  job  Ji  be  the  W-job  which  starts  at  or  before  e and  job  Jj  be  the  T-job  which 
starts  at  d.  Due  to  the  V-shape  structure  around  d,  job  J;  is  the  largest  W-job  and 
job  Jj  is  the  smallest  T-job.  Also,  no  W-job  can  be  longer  than  an  E-job.  If  schedule 
<7  does  not  satisfy  Equation  (A.l),  we  must  have  pi  > pj.  We  distinguish  the  cases 
I2kew(<r)  Pk-  Pi  +Pj  > d - e and  J2kew(<?)  Pk  ~ Pi  + Pj  <d  - e: 

(a)  'EkeW(v)  Pk  ~ Pi  + Pj  > d - e: 

In  this  case,  obtain  schedule  a ’ by  interchanging  jobs  J3  and  J;  such  that  Cj(a')  ~ 
Ci(a)  holds  (Figure  A.l).  We  have 

cw(Ji,  a)  = cw(W,  o)  = n(E(o))a  = cw(W,  o')  = cw(Jj,o') 

cw(Jj,o)  — n(T(o))f3  = cw(Ji,o') 
cw(Jk , a)  = cw(Jk,  o')  for  Jk  <£  {Jj,  J,} 

Applying  Equation  (2.7)  of  Lemma  3,  we  obtain 

cost(o')  - cost(o)  = ( cw(Jj,o ')  - cw(Jj,o))pj  + ( cw(Ji,o ')  - cw(Ji,o))pi 

= ( n{E(o))a  - n(T(o))(3)(pj  - pi)  < 0, 
since  pj  < pi  and  n(E(o))a  > n(T(o))/3  by  Lemma  2. 

Obtain  schedule  a"  from  o'  by  iteratively  sequencing  adjacent  jobs  starting  before 
d in  LPT  order  and  the  T-jobs  in  SPT  order.  Similiarly  to  the  proof  in  Lemma 
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e d 


i 

Jl 

Jj 

n(E(cr))a  • • • n 

e < 

1 

( n<r))p  ... 

i 

Jj 

Ji 

n(E(a))a  ■ ■ ■ n(T(a))/3 

Figure  A.l.  Ylkew(<r)  Pk  ~ Pi  + Pj  > d - e 


1,  parts  (c)(i)  and  (c)(ii),  we  have  cost(cr")  < cost(a').  If  max{pj  : Jj  G 

W{a' ')}  < min{p;  : J/  G T(<j")},  schedule  a"  satisfies  Equation  (A.l)  and  we 
are  done.  Otherwise,  we  proceed  as  follows:  If  Y,jk£W(o")  Pk  ~ ma XjjeW(a")  Pj  + 
m^nJteT(a")  Pi  > d — e,  we  repeat  the  above  process.  After  a finite  number 
of  iterations,  we  obtain  a schedule  a+  which  either  satisfies  Equation  (A.l)  or 
J2jkeW(<r+)Pk  - maxjjevv(a+)Pi  + minJ(e2>+)  Pi  < d - e holds.  In  the  former 
case,  we  are  finished,  and  in  the  latter  case,  we  proceed  with  part  (b). 

(b)  T,kew(a)  Pk  ~ Pi  + Pj  < d - e: 

In  this  case,  obtain  schedule  a'  by  interchanging  jobs  J2  and  J\  such  that  Sj(a’)  = 
e holds  (Figure  A. 2).  We  have 

cw(Jk,(r)  = cw(W,  <j)  = n(E(a))a  for  J k G W(a) 

cw(Jk,  cr')  = cw(W,  a1)  - n(T(a))f3  for  Jk  G W{a) 

cw(Jk , cr)  = cw(Jk,  a')  for  Jk  G E(a)  U T(cr) 

Applying  Equation  (2.7)  of  Lemma  3,  we  obtain 

cost(a')  - cost(cr)  = { ^ pk  - (d  - e)}(n(T(a))/3  - n(E(a))a)  < 0, 

Jk£W(o) 

since  J2jkeW{a)  Pk  > d - e and  n(T(a))fd  < n(E(a))a  according  to  Lemma  2. 
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e d 


1 

Jl 

Jj 

n(E(cr))a  n(T(a))/3 

d 

i _ 

Jj 

Jl 

n(T(a))(3  n(T(a))/3 

Figure  A. 2.  Ekew(c)  Pk  - Pi  + Pj  < d - e 


Now  obtain  schedule  a"  by  sequencing  jobs  with  Ck  > e in  SPT  order.  Notice 
that  a"  satisfies  Equation  (A.l)  since  ma ^-jk^w(<7")Pk  < pi  — ™injkeE(a)Pk  = 
minAeE(a")  Pk- 

2.  Ck  = e for  some  job  Jk  with  a V-shape  around  e: 

This  part  is  essentially  symmetric  to  the  case  Ck  — d.  For  the  construction  of  schedule 
o',  we  interchange  jobs  Jj  and  J/  such  that  Sj(a')  = Si(a).  Also,  schedule  a1  needs 
to  satisfy  5[!]  > 0 after  exchanging  the  two  jobs.  If  necessary  (5[j]  < 0),  both  the  due 
window  and  the  jobs  need  to  be  shifted  to  the  right  by  -5[X](>  0)  which  does  not 
affect  the  objective  function.  The  remainder  is  symmetric  to  the  case  Ck  = d. 

Finally,  since  the  jobs  are  indexed  in  SPT  order,  by  definition  of  nw*  and  Equation  (A.l), 
we  conclude  that  an  optimal  schedule  a exists  such  that  W(cr)  = { Jx, . . . , Jnw*}  holds. 

QED 


APPENDIX  B 


ALGORITHM  FOR  m = 2 PARALLEL  MACHINE  CASE 

Step  la.  Calculate  Cumulative  Weights: 

(ne,  nt,  index ) = (0, 0,  n ) 

(£\£2,r\T2)  = (0,0,0,0) 

WHILE  index  > nw  DO 
IF  (ne)a  < (nt  + l)(3  THEN 

(cwindex,  cwindex-1  = (nea,  nea) 

(E1,  E2)  <—  (E1  U {index},  E2  U {index  - 1}) 
index  = index  — 2 

ELSE 

(cw index i CW index— 1 = ((nt  T 1 )/3,  ( Tii  + l)/3) 

(Tl,T2)  <—  ( T 1 U {index},  T2  U {index  - 1}) 
index  = index  — 2 

Step  lb.  Calculate  Cost  for  E-jobs  and  T-jobs: 

ETnw+ 3 — 5Zj=ntii+ 3 cu,j  * Pj  2 = ET™w+3  + CWnw+ 2 * Pnw+ 2 

Step  lc.  Calculate  A7""-1  and  A"™: 

a™-1  = E"=rJ  Pi  a™  = a"-1  + Pnu; 

Step  2a.  Initialization  of  cost™-: 

r rmn 

cos^min  = °°»  ^77i2?i  an<l  P°s^in  need  not  be  initialized. 
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Step  2b.  Using  the  results  of  the  DP,  examine  all  schedules  with  nw  Wl-jobs: 
FOR  ALL  Jf.  £ { J\i  • . . , Jnw+ 1}  DO 

(Lower,  Upper ) = ((Anw+1  - Pk  - (d  - e))+,  [(A"",+1  - pk)/ 2J) 

(note  the  symmetry  for  t > Upper ) 

FOR  t = Lower  TO  Upper  DO 
IF  fk(nw  + 1 ,t)  = YES  THEN 

Examine  both  cases  for  job  Jk  (machine  M 1 or  M2) 

IF  t + pk  > d - e AND  (Anw+1  - pk  - t)  + pnw+2  > d-e 
(i.e.,  Jk  £ JS 1 1 Jnw+2  £ JS2,  and  n(WL)  = nw)  THEN 

«)) 


e)) 


COSt  1 — CWnw-k\  * (t  T Pnw-\-2  (d  &)) 


cost  1 =: 

cwnw^.\  * (t  -f  pk  (d  e)) 

+ 

CWnw+ 2 * ((AnU;+1  - Pk  ~ t)  + Pnw+2  ~ ( d 

+ 

TTTn 
£ylnw+ 3 

cost2  = 

CWnui+2  *(t  + pk-(d-  e)) 

+ 

CUWH  * ((A™*1  - - /)  + p„w+2  - (d 

+ 

jprpn 

rjlnw+3 

update  cosC“„,  CC>  and  position  of  Jk'- 
IF  costl  < cost™in  THEN 

CO$inW-  — 1 * fTlW  — 4.  PopTlW  — /l/l 

IF  cosf2  < cosCC  THEN 

c“C  = cost2;  Cm  = ^Cm  = M 2 
IF  t + p-nw+2  > d - e AND  (Anw+1  - pk)  - t + pk  > d - e 
(i.e.,  Jnw-\- 2 £ JS1,  Jk  £ JS2,  and  n(VU7)  = nw)  THEN 


+ cwnw+ 2 * (((Anu;+1  -pk)~t  + pk)  -(d-  e)) 
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+ ETL+3 

COSt2  — CWnw-\- 2 * 4"  Pnw+2  ^)) 

+ cwnw+ 1 * (((Anu;+1  - Pk)  - t + Pk)  -(d-  e )) 

+ EEnw+3 

update  cost™in,  t™in,  and  position  of  Jk: 

IF  costl  < cost™m  THEN 

rntfnw-  — inw.  — i-  Pnsnw  — M1 

cuslmin  ~ ~ rosmm  ~ M 

IF  cost2  < cost™in  THEN 

cost™n  = COSt2-  C7n  = t;  Pos™tn  = M2 

Step  3.  Using  the  results  of  the  DP,  examine  all  schedules  with  nw  — 1 Wl-jobs: 
This  is  done  similiarly  to  Steps  2a  and  2b. 


Step  4.  IF  costZn  < cost™-1  THEN 

(the  optimal  schedule  has  nw  Wl-jobs,  processes  Wl-jobs  in  machine  M 1 with  a cu- 
mulative processing  time  of  t™n,  and  positions  job  Jnw+\  in  machine  Pos™n) 

We  now  calculate  the  total  processing  time  PE ‘ (PT‘)  of  the  jobs  in  machine  M' 
which  are  processed  before  (after)  the  Wl-jobs  in  machine  M‘. 

For  the  allocation  of  the  jobs  to  machines  M 1 and  M2,  we  basically  use  the  sets  El 
and  Tl  which  were  generated  in  Step  la.  Yet  we  need  to  distinguish  two  cases  n — nw 
is  even  and  n - nw  is  odd.  We  also  calculate  minimum  values  e*m8„  and  dimin  for  the 
location  of  the  due  window.  The  due  window  must  satisfy  d > dimin  and  d > eimin-\-K 
for  all  i G {1,2}. 

If  n — nw  is  even,  the  jobs  Jnw+i  and  JnU;+2  are  allocated  to  the  machines  according 
to  the  calculation  in  Step  3. 

If  n — nw  is  odd,  we  need  to  allocate  jobs  Jnw+ 2 and  Jnw-\. 3 at  the  same  positions  on 
different  machines.  First,  we  handle  the  case  n — nw  is  even. 
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IF  n — nw  is  even 
FOR  i = 1 TO  2 DO 


EE  YljeE'r\{nw+3,...,n}Pj  PP  — YljeT'n{nw+3,...,n}Pj 


IF  Pos™in  = M1  THEN 


IF  (nw  + l)  e E1  U E2 

THEN  PE1  - PE1  + pnw+l 
ELSE  PT1  - PT1  + pnw+1 

IF  (nw  + 2)  € E1  U E2 

THEN  PE2  <-  PE2  + Pnw+2 
ELSE  PT2  <-  PT 2 + Pnw+2 


-/I  . — ppll  fUW 

ulmm  ~ 1 \ lmin 


^1  min  — PE 


d2mtn  = PE 2 + (A™  - tZn) 


po  . — pp2 


ELSE  (Pos™in  = M2) 


IF  (nw  + 1)  € E1  U E2 

THEN  PE2  - PE2  + pnw+1 
ELSE  PT2  <—  PT2  + pnw+\ 
IF  (nw  + 2)  € E1  U E2 

THEN  PE1  <-  PE1  + Pnw+2 
ELSE  PI 1 <—  PT1  + Pnw+2 


d2min  = PE2  + (Anw  - C?») 


e2miri  = PP2 


eflr 


PE1  + t™n 


pi  — PF1 

C'±min  — 1 ej 


The  due  window  [e,d]  satisfies  eimi7l  > e and  dimin  > d.  Therefore,  we  have 

d max(dlml'n,  d2min,  el min  T E i ^2mjn  f A ) 

This  defines  the  location  of  the  due  window.  The  starting  times  of  the  jobs 
obtained  accordingly. 


IF  n — nw  is  odd 

In  this  case,  the  initialization  of  PE 1 and  PT1  starts  from  nw  + 4: 


are 


FOR  i — 1 TO  2 DO 
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S jEE‘n{nw+4,...,n } Pj 
— Xvj£T'n{nui+4,...,n}  Pj 

The  case  distinctions  are  identical  to  the  previous  case.  In  addition,  we  take  the 
following  action:  Whenever  we  allocated  job  Jnw+ 2 in  the  previous  case  (n  — nw  is 
even),  we  now  (n  - nw  is  odd)  allocate  Jnw+ 3 to  the  other  machine. 

Step  5.  IF  costZn  > cost™-1  THEN 

(the  optimal  schedule  has  nw  — 1 Wl-jobs,  processes  Wl-jobs  in  machine  M 1 with  a 
cumulative  processing  time  of  t™~ 1,  and  positions  job  Jnw  in  machine  Pos™*1) 
The  procedure  is  similiar  as  in  Step  4. 


APPENDIX  C 

CALCULATION  FOR  A PRODUCTION  SYSTEM  WITH  PERIODIC  MAINTENANCE 


C.l  The  range  of  5i(arfOD  for  Problem  P^  (Case  B4-2) 

The  goal  of  this  section  is  to  prove 

5i(x(0))  < Cr  < a,  for  a;(0)  € [d(a  - Cr ),  d(a  + ^)], 

and 

S\(d(a  - Cr ))  = CT  < a. 

We  proceed  by  showing  that  the  function  S'i(a;(0))  is  monotone  decreasing, 
^T^Si^O))  < 0,  for  a:(0)  £ (d(a~  Cr),d(a  + ^)), 
and  that  the  function  value  at  x(0)  = d(a  — Cr)  is  Cr , 


Si(d(a  — Cr))  = CT. 


Since  we  have 


at  t c\w  x(0)  - da  + Ub  x(0)-da  + Ub.„  2 b 6X  . .. 

s.um  = -uu+i  + '/(-u-„+<i  );  + cTm«« + 5)  - *(»))• 


(see  Equation  (4.77)),  we  have  for  the  first  derivative 
d 


dx(  0) 
1 


S i(*(0)) 


+ 


x(0)-da+Ub\  1 2b 

U+d  > U+d  U+d 


u + d + toW-  + 1)  - »(<>)) 


1 


U + d 


( a^(0) — da+Ub  ^ L 

1 U+d  ) ~ 0 


1) 


1 


U + d 


\/(£i2te^)2  + ^(«  + |)-^(0)) 

(x(0)  — d(a  + b )) 

y/(x(0)  — da  + Ub)2  + 2 b{U  + d)(d(a  + §)  - *(0)) 


-!)• 


(C.l) 

(C.2) 


(C.3) 
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Regrouping  the  radicant,  we  obtain 

(x(0)  - da  + Ub )2  + 2b(U  + d)(d(a  + ^)  — x(0)) 

= (x(0)~  da)2 + 2(x(0)-da)Ub  + (Ub)2 

-2(a;(0)  — da)Ub  - 2(x(0)  — da)db  + ( db )2  -f  Udb 2 
= (x(0)  - da)2  - 2(x(0)  - da)db  + (db)2  + (Ub)2  + Udb2 
= (x(0)-d(a  + b))2+U(U  + d)b2 
> (x(0)  - d(a  + b))2. 


Together  with  Equation  (C.3),  this  yields 


d 

dx(  0) 


5i(x(0)) 


1 , (ar(0)-d(a  + 6)) u 

u + d[  y/(x(0)  - d(a  + 6))2  + U(U  + d)b 2 ’ 

< 0,  for  x(0)  G [d(a  - Cr),  d(a  + ^)). 


It  now  remains  to  show 


S(d(a-Cr))  = Cr. 


(C.4) 

(C.5) 


(C.6) 


S(d(a  - Cr))  - C, 
Ub 


dCr 


U + d 


+ 


Ub-dCr.„  2b  ,b  ^ „ 

U + d } + U + d{d 2 + dCV) 


Eh  - dCr 
U + d 


\ « 26  6 Ub-dCr, 

)2+uTdid2+dCr)-{c'+-u^r) 


(^)2  + + <*?r)  - {cv  + ^}2 


E/+rf 


\A 


Ub-dCr  \2  I Jib  (fib  i Jf  \ , ( U , Ub  \ 

u+d  I + u+dya2  + + \u+dUr  + u+d) 

26 

t7+dv 


*(4 + dcr)  ~ a - 2 


v'(BTOi)2  + TO(4  + ^)  + {jj+ct  + Si) 

(_1  1 Jid_\/~i2  1 / 2bd  _ 2Ub  \/~(  . t7— ei  d62 

v 1 u+d)^r  "r  t[/+d  [7+d/°r  ' 


t/+rf  17-rf 


\f+Wy+^A+^) + (toc, + $,) 


(7  - d 


c?  + 26 C,  - Si 


u + d Vfw1)2  + uh(4  + <«?r)  + (p^a  + ^3) 


= 0, 


according  to  Equations  (4.41)  and  (4.45). 

In  conclusion,  we  have  verified  Equation  (C.l). 
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C.2  Switching  Point  for  Problem  Pi  fCase  B4-2) 

In  this  section,  our  goal  is  to  verify 

x*(a  — 5i(x(0)))  > 0,  for  z(0)  £ (d(a  — Cr),  d(a  + ^))  (C.7) 

(compare  Property  (4.29)).  Since  the  switching  point  satisfies 

x*(a  - Si(x(0)))  = x(0)  - d(a  - 5i(a:(0))), 
we  define  a function  f(x( 0))  as  follows, 


/(*(0))  = *(0)-rf(a-51(a(0))). 

We  verify  Equation  (C.7)  by  showing  that  the  function  is  monotone  increasing  for  the 
interval  (d(a  — Cr),d(a  + |))  and  has  a root  at  d(a  — Cr ).  Therefore,  we  first  show  that  the 
derivative  of  f(x( 0))  with  respect  to  x(0)  is  positive.  Since  we  have 

s4_/(x(0))=l+rfs4_ft(x(0)), 

and 

i+rf^,(i(o))>i+',4i)Si(x(o)) 

due  to  ^^^(^(O))  < 0 (see  Equation  (C.5)),  it  suffices  to  show 

i + 4i)s,(i(o))  > °-  (c-8) 


From  Equation  (C.4),  we  obtain 


d 


dx(  0) 
1 


5i(x(0))  + 


1 U + d 


(- 


U + d U 
(a:(0)  - d(a  + b )) 


d. 


U + d V(*(0)  - d(a  + b))2  + U{U  + d)b 2 
1 <V(s(0)  - d(a  + 6)) 2 + U(U  + d)b*  - U(d(a  + b)  - x(0)) 

U + d £V(*( 0)  - d(a  + b ))2  + U(U  + d)b 2 

1 d2{(x(0)-d(a  + b))2  + U(U  + d)b2}  - U2(d(a  + b)  - x(0))2 


U + d 


Uy/{x(0)  - d(a  + b))2  + U(U  + d)b 2 

1 


<V(x(0)  - d(a  + b))2  + U(U  + d)b2  + U{d{a  + b)  - z(0)) 
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1 U(U + d)d2b2 -(U2 -d2)(x{ 0)  - d(a  + b))2 
U + d Uy/{x( 0)  - d(a  + b))* *  + U(U  + d)b 2 
^ 1 

*dx/(x(0)  - d(a  + b)Y  + U(U  + d)b 2 + t7(rf(a  + 6)  - *(0)) 

C/2_d2  -{(*(o)-d(a  + &))2-^d2&2} 

f/  + rf  Uy/(x(0)-d(a  + b))2  + U(U + d)b2 
* 1 

* dy/(x(0)  - d(a  + 6))2  + U(U  + d)b 2 + IT(rf(o  + 6)  - *(0)) 

(u  _ d)  -{(^(O)  ~ + b))  - d(Cr  + 6)}{(x(0)  - d(a  + b ))  + d(Cr  + b )} 

\A*(0)  - d(a  + 6))2  + U(U  + d)62 

1 

*<V(a:(0)  - % + b))2  + U(U  + d)b 2 + C7(d(a  + 6)  - *(0)) 

(u  _ J,-{(x(0)-d(a  + 2b  + Cr))}{(x(0)  - d(a  - Cr))} 
tVU(O)  - d(a  + b))2  + U{U  + d)62 

^ 1 

*<V(*(0)  - d(a  + b)Y  + 17(17  + d)b 2 + £7(rf(a  + 6)  - *(0)) 

> 0 

for  *(0 ) £(d(a-Cr),d(a+^)).  (C.9) 


Thus,  we  have  validated  Equation  (C.8), 

^oj/(*(0))  > 0,  for  x(0)  G (d(a  - Cr),  d(a  + ^)). 
Therefore,  we  have 


min{/(®(0))  : *(0)  € (d(a  - Cr),  d(a  + ^))}  = f(d(a  - Cr)). 


By  Equation  (C.6),  we  have 


f(d(a  - Cr ))  — d(a  - Cr)  - d(a  - Cr)  = 0. 

This  verifies  Equation  (C.7). 

C.3  The  Switching  Curve 

It  has  been  observed  in  Section  4.3.4  that  the  optimal  trajectories  which  have  a switching 
point  do  not  touch  the  trajectory  Xq{1)  during  the  interval  [0,a]  (see  Equations  (4.103)  and 
(4.104)). 
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Figure  C.l.  Behavior  of  Type-2  Structures. 
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In  this  section,  we  explain  this  phenomenon  from  a different  perspective.  Figure  C.l 
shows  two  trajectories.  In  part  (a)  of  Figure  C.l,  we  have  a trajectory  for  x(0)  G ( d(a  — 
CT ),  da)  with  a switching  point  at  a-ST  and  in  part  (b)  of  Figure  C.l,  we  have  the  Trajectory 

*o(<)- 

Sr  in  part  (a)  of  Figure  C.l  is  given  in  such  a way  that  the  switching  point  lies  on  the 
Trajectory  ^(f).  The  defining  equation  is 


1(0)  - d{a  - Sr)  = {U  - d)((a  - ST)  - (a  - Cr)).  (C.10) 


This  results  in 


Cr 


x(0)  — d(a  - CT) 
U~ 


(C.ll) 


We  now  express  the  performance  measure  for  the  trajectory  in  part  (a)  in  terms  of 
ST  and  for  the  trajectory  in  part  (b)  in  terms  of  Cr.  The  performance  measure  for  the 
trajectory  in  part  (a)  is 


Jl(Sr) 


= r 5r(z(0)  - dtfdt  + /“  (z(0)  - d(a  - Sr)  + (U  - d)(t  - (a  - Sr)f 
Jo  J a — Sr 

+ [a+b(x(0  )-dt  + USr)2dt 

J a 

= [ Sr(x(0)-dt)2dt+  r (x(0)~  Ua  + USr  + (U  - d)t)2dt 

JO  J a— Sr 

+ [a+b(x(0)~  dt  + USr)2dt. 

J a 


dt 


We  now  want  to  demonstrate  that  the  trajectory  given  in  part  (a)  is  not  optimal.  For 
this  reason,  we  replace  Sr  by  a parameter  5,  take  the  derivative,  and  evaluate  the  derivative 
for  S = Sr.  We  have 

JgMS)  = Ja  s'2U(x(0)-Ua  + US  + {U-d)t)dt  + J“+b2U(x(0)-dt  + US)dt. 

We  now  rewrite  the  expressions  (ar(0)  - Ua . + US  + (U  - d)t)  and  (x(0)  - dt  + US).  With 
S = Sr  = Cr-  , we  have 

*(0)  - Ua  + US  + {U  - d)t  = .r(0)  - Ua  + UCr  - x(0)  + d(a  - CT)  + (U  - d)t 

= (U  - d)(t  - (a  - Cr)), 


and 
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z(0 ) — dt  + US  = x(0)  — dt  + UCr  - x(0)  + d(a  — Cr) 


(U  - d)Cr  - d(t  - a). 


Therefore,  we  obtain 


2 U(U-d)tdt  + / 2 U((U  - d)Cr  - dt)dt 


U{U  - d){Cl  - (CT  - Sr  f + 2Crb  - db2} 


-U(U  - d){(Cr  - Sr)2} 


(C.12) 


< 0. 


(C.13) 


Thus,  the  trajectory  in  part  (a)  is  not  optimal  and  the  optimal  value  S*  satisfies  S*  > Sr 
according  to  Equation  (C.13).  The  optimal  switching  time  a - S*  is  therefore  smaller  than 
a - Sr  and  the  resulting  optimal  trajectory  will  not  touch  the  trajectory  Xg(t). 

We  come  to  the  same  conclusion  when  we  consider  parts  (a)  and  (b)  qualitatively.  We 
know  that  Trajectory  x^(t)  is  optimal  for  x(0)  = 0.  This  implies  ^Ji(Cr)  = 0.  We  can 
partition  the  performance  measure  in  two  parts  so  that  Ji(C)  = Ja(C)  + Jb(C)  where  Ja(C ) 
refers  to  the  interval  [0,  a — 5V]  and  Jb(C)  refers  to  the  interval  [a  — 5r,  a + b]: 


a-C 

C-Sr 


(C.14) 


Jb(C)  = Ji(C)-Ja(C) 


According  to  Equation  (C.14),  it  is  not  difficult  to  see  that 


-^Ja(C)  = (U  - d)2(C  - Sr)2  > 0. 


(C.15) 


Therefore,  we  have  for  C — Cr, 


(C.16) 
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From  parts  (a)  and  (b)  of  Figure  C.l,  notice  that  the  derivative  of  J&(C)  for  part  (b) 
at  C — CT  is  proportional  to  the  derivative  of  Ji(S)  for  part  (a)  at  S = Sr.  To  see  this, 
consider  the  areas  A\  to  A6  in  Figure  C.l.  The  triangles  A 2 and  A5  as  well  as  A3  and  Ae 
correspond  to  each  other.  Polygon  Ai  is  contained  in  triangle  .4 4.  The  time  interval  which 
is  covered  by  Ai,  A 2,  and  A3  is  [a  — Sr,  a -f  b]  and  the  time  interval  which  is  covered  by  A4, 
A5,  and  Ae  is  [a  — Cr,a  + b ].  Therefore,  the  change  in  the  performance  measure  for  part 
(b)  for  the  interval  [a  — Sr,a  + 6]  (^Jb(Cr))  is  proportional  to  the  change  in  performance 
measure  for  part  (a)  -Ji(Sr )). 

Quantitatively,  this  observation  is  verified  by  Equations  (C.12),  (C.15),  and  (C.16): 

{u  ~ d)JsJl{Sr)  = ~u^cJa{Cr)  = u^cMCr)- 

And  since  Jb(CT ) is  negative,  j§Ji(Sr)  is  negative  as  well.  But  this  in  turn  implies 
that  the  parameter  S must  be  increased  and  that  the  optimal  value  S*  must  satisfy  S*  > Sr. 
Consequently,  the  optimal  trajectory  for  z(0)  > d(a-Cr)  will  not  touch  the  Trajectory  Xg(t). 
We  have  thus  shown  that  our  quantitative  conclusion  can  also  be  observed  qualitatively  by 
Figure  C.l. 


C.4  The  First  Derivative  of  JjTadO)) 

The  first  derivative  of  Jj(a:(0))  with  respect  to  z(0)  is  calculated  from  Equation  (4.105) 
according  to  the  given  ranges. 

Case  El:  i(0)  < — (U  — d)(a  - Cr ). 

^y<A*(*(0))  = Joa2(x(0)  + (U -d)t)dt  + J“+b2{x{0)+Ua-dt)dt 

= 2x(0)a  + {U-  d)a 2 + 2x(0)b  + 2Uab  - d((a  + b)2  - a2) 

= 2a:(0)(a  + b)  + (U  - d)(a2  + 2 ab  + ( b 2 - b2))  - db 2 


*(0)  U u2 


— (U  — d){ 2— — -( a + 6)  + (a  T b)2  — 


U-d 

Case  E2:  or(0)  € (-(U  - d)(a  - Cr),0). 
d 


U-d 


b2}. 


(C.17) 


dx(  0) 
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0 + 2z(0){-^%  + (t/-d){ 


U -d 


~»(0)12 
U -d] 


*2(0) 

U-d‘ 


(C.18) 


Case  E3:  z(0)  G (0,  d(a  - Cr)). 
d 


dx{  0) 


T(f)\  f ' 

^i(*(0))  - d{x{0)~d{d})2  + L 2(x(°)~dt)dt 
= o + 2.(0  )m}-dA> 


*2(0) 

d ' 


(C.19) 


Case  E4:  z(0)  € (d(a  — Cr ),  d(a  + |)). 

Due  to  the  length  of  the  calculation  for  this  case,  we  calculate  its  derivative  in  Appendix 
C.5.  We  have 


/f(x(0))  = 2x(0)(a  + 6)  — d(a  + b)2  + U Sf(x(0))  + 2U  Si(x(0))b.  (C.20) 


dx{  0) 

Case  E5:  x(0)  > d(a  + |). 

d 


imr'(xm  = C 2(x(0)  ~ dl)dt 

= 2x(0)(a  + b)~  d(a  + b)2. 


(C.21) 


This  completes  all  cases  for  the  range  of  x(0).  The  result  is  summarized  in  Equation 
(4.106). 

C.5  The  Derivative  of  JjTadOP  (Case  E41 
In  this  section,  we  calculate  ^q)T1*(x(0))  for  z(0)  G {d(a  - Cr),d(a+  |)). 

^*(x(0)) 


dx(  0) 


da;(0) 


<x 


a — S\  (:r(0)) 


(a:(0)  — dt)2dt 


fSi(x(0)) 

+ / (*(0)  - d(a  - 5i(*(0)))  +(U  - d)t)2dt 

J 0 
ra+b 

+ (x(0)~  dt  + USrix^fdt} 
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{ -^o)*(*(°))H*(°) _ d(a ~ ^i^0)))}2 

ra-S1(x(0)) 

+ / 2(x(0)  — dt)dt 

Jo 

+{^oj5l(a;(0))}{a;(0) " d{a  - W0)))  + (u~  <^i(*(0))}2 

/•5i(a;(0))  j 

+ yo  2{x(0)  - d(a  - S1(x(0)))  + (U-  d)t}{l  + d-—S1(x(0))}dt 
+ [a+b  2{x(0)-dt  + US1(x(0))} 

J a 

{l  + ud7( Sjs>W°))}<“ 


ra—Si  (r(0)) 

{ / 2(x(0)  — dt)  dt 

Jo 

rSt(x(0)) 

+ 2{x(0)  - d(a  - S1(x(0)))  + (U  - d)t}  dt 

J 0 

+ [a+b  2{x(0)  + USi(x(0))  - dt}  dt} 

J a 


4 

{-{*(0)  - d(a  - 5i(x(0)))}2 
+ {*(0)  - d(a  - Si(x(0)))  + (U  - d)S1(x( 0))}2 
+d{ 2{(*(0)  - d(a  - Si(x(0))))Si(s(0))}  + (U  - d)Sj{x( 0))} 

+^{2{(x(0)  + U 5!(x(0)))6}  — d((a  + b)2  — a2)}  }.  (C.22) 


We  will  now  consider  the  factor  which  multiplies  ^y5i(x(0))  in  Equation  (C.22): 


{-{x(0)-d(a-5!(x(0)))}2 
+{x(0)  - d(a  - S1{x{ 0)))  + (U  - d)S1(x( 0))}2 


+d{2{(x(0)  - d(a  - 5i(*(0))))51(x(0))}  + (U  - d)S}(x( 0))} 
+U{ 2{(x(0)  + US1(x(0)))b}  - d((a  + b)2  - a1)}  } 


(2{x(0)  - d(a  - Si(z(0)))}{(£7  - d)S1(x( 0))}  + {(tf  - d)Si(*(0))}2 
+^{2{(z(0)  - d(a  - 51(x(0))))5i(a;(0))}  + (E7  - d)S2(*(0))} 

+ [/{2{(.r(0)  + U 5x(a:(0)))6}  - d((a  + bf  - a2)}  } 
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2x(0)tfS1(x(0))  - 2*(0)d5i(®(0))  - 2aUdS1(x(0))  + 2ad2S1(z(0)) 
+2U dS2(x(0))  - 2d2Sf(x(0)) 


+2®(0)d5i(x(0))  - 2d2aS1{x{0))  + 2d2S12(x(0))  + UdSf(x{ 0))  - d2Sf(x( 0)) 
+2x(0 )Ub  + 2U2bS1{x{0))  - 2abUd  - b2Ud 

{2Ud  - 2d2  + U2-  2 Ud  + d2  + 2d2  + Ud-  d2}S2(x(0)) 

+ {2x(0)U  - 2x(0)d  - 2 aUd  + 2 ad2  + 2x(0)d  - 2d2 a + 2U2b}S1(x(0)) 
+{2x(0)Ub  - 2abUd  - b2Ud} 

{ U 2 + Ud}S2(x(  0)) 

+{2x(0)17  - 2aU d + 2U2b}Sl(x(0)) 

+{2x(0 )Ub  - 2 abUd  - b2Ud } 


according  to  Equations  (4.41)  and  (4.77).  Thus,  substituting  the  result  of  Equation  (C.23) 
into  Equation  (C.22),  we  have 


+U2S2(x(0))  - 2UdSl(x(Q))  + d2Sl(x( 0)) 


{U(U  + d)}{S2(x(  0))  + 2 


x(0)  — da  + Ub 
U + d 


sii*m + - d(a + ^)) 


0, 


(C.23) 
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+ 

+ 


l 

l 


Si(x{0)) 

a+b 


</o 

/: 

L 
l 

f 

F 

J a 


a— Si  (i(0)) 


Si(x(0)) 

a+b 


a+b 

Si(*(0)) 

ra+b 


2{x(0)  - d{a  - S!(x{0)))  + (U  - d)t}  dt 
2{x(0)  + U 5i(:r(0))  — dt}  dt} 

2(a;(0)  — dt)  dt 

2{x(0) -dt  + U{t- (a -S^O))))}  dt 
2{z(0)  - dt  + t/SiOr(0))}  dt} 

2(x(0)  — dt)  dt 
2Ut  dt 

2US1(x(0))  dt} 


2z(0)(a  + b)  - d(a  + b )2  + t/Sft^O))  + 2t/S1(z(0))6.  (C.24) 


C.6  The  Second  Derivative  of 

According  to  Equation  (4.106)  (see  also  Cases  El  to  E5  in  Appendix  C.4),  we  obtain 
the  following  expressions  for  the  second  derivative  of  Jj*(x(0)). 

Case  FI:  x(0)  < -( U - d)(a  - Cr). 

In  this  case,  we  have  from  Equation  (4.105)  according  to  Leipniz’  rule 


d2 


dx2( 0) 


./r(z(o))  = 


d fa  ra+b 

^ 2(x(0)  + (U  - d)t)dt  + J 2(x(0)  + Ua-  dt)dt} 

ra  ra+b 

= 2 dt+  2 dt 

Jo  Ja 

= 2(o  -f  b).  (C.25) 


Case  F2:  x(0)  G (—(U  — d)(a  - Cr),  0). 
According  to  Equation  (4.105),  we  derive 


d2 

dx2( 0) 


jr(x(o)) 


U-d}) 
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r-j(°) 


U -d 

-2z(0) 

U -d 


+ 2(z(0)  + (U  - d)t)dt) 

Jo 

)+r 


0-  -2(x(0)  + (U  - d)  a’(0) 


— j(°) 
d 


U -d' 


2 dt 


(C.26) 


Case  F3:  a:(0)  € (0,  d(a  — Cr )). 
From  Equation  (4.105),  we  derive 


d2 


dx2( 0) 


jr(x(o)) 


d 


dx(  0) 

°+^ 

2®(0) 


{d{x{0)~d{d})2  + 1 2(*(0)  - rff )<«} 


= 0+i2(i(0  )-<i(^))  + jf 


*(°) 

d 


2dt 


d ' 

Case  F4:  x(0)  £ (d(a  - Cr),  d(a  + |)). 

In  Appendix  C.5,  we  derive  from  Equation  (4.105) 

4^‘wo)) 

= tTPrft”- W))-*(°)^oj 


<S'i(*(0)) 


+rf(“+t4+4))5,W0)))}- 


Case  F5:  x(0)  > d(a  + |). 


In  this  case,  we  have 


d2 


dx2( 0) 


•*?(*(  0)) 


dx(  0) 

/•a+6 


{ J 2(x(0)  - dt)dt} 


= r 2dt 

Jo 

= 2(a  + 6). 


(C.27) 


(C.28) 


(C.29) 


This  completes  all  cases  for  the  range  of  a:  ( 0 ) . The  result  is  summarized  in  Equation 


(4.109). 


C.7  A Property  of  JjTxf 0) ) 

In  this  subsection,  we  want  to  validate  Equation  (4.107).  That  is,  we  want  to  show 
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ajjoy^ifao)  < 0,  for  x(0)  < 0, 

^ojjr(xo)  = 0,  for  x(0)  = 0, 

4oyJr(x0)  > 0,  for  x(0)  > 0. 

We  proceed  by  considering  the  ranges  for  x(0)  according  to  Cases  Cl  to  C5  (see  Equation 
(4.105)). 

Case  Gl:  x(0)  < — (17  — d)(a  - Cr ). 

According  to  Equation  (C.17),  we  have 

^(0)^(0))  ~ (U  ~ + b)  + (a  + b)2  - (C.30) 

and  conclude 

SUP{^0)'7l*(X(0))  : X(0)  < ~(U  ~ ^ ~ Cr ^ = ("(^  “ d^a  - Cr))- 

We  therefore  have  for  the  left-hand-limit  of  ^y./j  (x(0))  at  x(0)  = -([/  - d)(a  - CT) 

hm 

x(0)^-(U-d)(a-CT),  x(0)<-(U-d)(a-CT)  «fx(0) 

— (U  — d){—2(a  — Cr)(a  + b)  -f  (a  + b )2  — — y&2} 

= (U  ~ d){—2(a  + b-  J jb)(a  -(-  b)  + (a  + 6)2  - ^ ^fr2} 

= (U-d){-(a  + bf  + 2]f^^b(a  + b)-^b>} 

= HU-d){((a  + b)-]j^^b)i} 

= -(U  - d){a  - Cr}2 

< o, 

and  conclude 

^LWO))  < 0,  for  x(0)  < -( U - d)(a  - Cr). 


Case  G2:  *(0)  € {-(U  - d)(a  - CT),  0). 


184 


According  to  Equation  (C.18),  we  have 

^•(*(0))  = -^i,  (C.31) 

and  conclude 

^5 y^i(*(0))  < 0,  for  *(0)  € (~(U  - d)(a  - Cr),  0). 

Case  G3:  x(0)  S (0,d(a—  Cr)). 

According  to  Equation  (C.19),  we  have 

o)) = ^ (c-32) 

and  conclude 

Ji*(a:(0))  > 0,  for  x(0)  € (0,d(a-Cr)). 

Case  G4:  a;(0)  € (d(a  - Cr),  d(a  + |)). 

In  the  same  way  as  in  Cases  Gl,  G2,  and  G3,  it  can  be  shown  that  the  derivative 
of  J*(x(0))  with  respect  to  x(0)  according  to  Equation  (4.105)  is  positive  for  the  range 
(d(a  — Cr),d(a+  |)).  It  turns  out  that  this  requires  tedious  calculation. 

Instead,  we  show  that 

-7T(*(0)  + Aar(0))  - Jf(*(0))  > 0, 

for  Ax(0)  G (0,  d(a  - Cr))  and  z(0)  € (d(a  - Cr ),  d(a  + - Ax(0))(C.33) 

For  simplicity,  we  denote  a;(0)  by  2q(0)  and  x(0)  + Ax(0)  by  2:2(0).  According  to 
Equation  (C.l),  we  have 

for  2:(0)  e [d(a  - Cr),d(a-\-  ^)], 


5'i(2:(0))  < Cr  < a, 
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d(a  + |) 
*2(0) 

zi(0)  = a;3(0) 


d(a  — Cr ) 


a — Cr 


a 


a + b 


Figure  C.2.  J^(x( 0))  is  monotone  increasing  for  z(0)  G ( d(a  - Cr),d(a+  |)) 

Since  zi(0),  x2(0)  € (d(a  - CT),  d(a  + |)),  we  have  according  to  Equation  (4.85) 

tti(0  = 0,  for  t e [0,a-Cr)  C [0,a-  5i(*i(0))), 

u*2(t)  = 0,  for  t £ [0,a-Cr)  C [0,a-  S1(x2(0))). 

The  optimal  trajectories  a;J(f)  and  x2(t)  during  [0,a  - Cr]  are  shown  in  Figure  C.2.  In 
order  to  compare  Jj‘(x1(0))  and  Jj*(a;2(0)),  we  consider  a third  trajectory  a;3 (/)  which  is 
obtained  as  follows:  We  start  with  an  initial  production  surplus  level  of  a;3(0)  = aq(0)  and 
use  the  production  control 


Thus,  the  trajectory  x3(t)  starts  at  the  same  initial  production  surplus  as  trajectory 
Xi(t),  stays  on  this  level  until  it  reaches  trajectory  x2(t)  and  during  [g2(°)-gi(°)^  jg 

identical  to  trajectory  x2(t).  Since  the  trajectory  .c3(t)  is  positive  and  below  trajectory 


u3(t)  = < 


d for  te  [0,  yM°)-^(°)); 


which  results  in  a trajectory  x3{t)  with 
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x\ (0  during  [0,  £2M_£ilo]j  an(j  subsequently  coincides  with  it,  we  conclude 

j xl(t)dt  < J{{x*2{t)).  (C.34) 

Since  trajectory  x\(t)  is  the  optimal  trajectory  starting  from  Xi(0)  = x3(0),  we  have 

(*iW)  < [T  x23(t)dt.  (C.35) 

J 0 

From  Equations  (C.34)  and  (C.35),  we  conclude 


^(*i(0))<  Jr(*2(0)). 


Consequently,  we  have 


^(,(»))=  Urn  gW)  + M0))-W0))>0 
dx( 0)  v ''  As(o)^o  Ax(0) 


(C.36) 


for  z(0)  € ( d(a  - Cr),  d(a  + |)).  Thus,  we  have  shown  in  a different  way  that  the  derivative 
for  this  range  of  initial  production  surplus  values  is  positive. 

Case  G5:  x(0)  > d(a  + |). 

According  to  Equation  (C.21),  we  have  for  z(0)  > d(a  + |) 


d 


dx{  0) 


Jf(x(0))  = 2x(0 )(a  + b)  - d(a  + bf 


We  conclude  for  z(0)  > d(a  + |), 


Ji*(x(0))  > 2 d(a  + ^)(a  + b)  - d(a  + b)2 
= d{2a  + b)(a  + b)  — d(a  + h)2, 
= da(a  + b), 

> 0. 


Finally,  we  observe 


lim 


(o)-+o,  ®(o)>o  dx( 0) 


Jj(x(0))  = lim 


d 


(0)-^0,  a-(0)<0  dx( 0) 


(*(o))  = 0. 


(C.37) 


(C.38) 


187 


In  summary,  we  have  validated  Equation  (4.107) 

d^a)J\ixo)  < 0,  for  *(0)  < 0, 

a^gj^f(*o)  = 0,  for*(0)  = 0, 

4o)Ji(x o)  > 0,  for  *(0)  > 0. 

C.8  A Property  of  jjjQjJZ  (a:(0)) 

In  this  subsection,  we  want  to  vahdate  Equation  (4.188).  That  is,  we  want  to  show 

S^b)  JZ(X°)  < °>  for  *(°)  < 0, 

d^o)J2(xo)  = 0,  for  ar(0)  = 0, 

d$o)JZ(xo)  > 0,  for  *(0)  > 0. 

We  proceed  by  considering  the  ranges  for  x(0)  according  to  Equation  (4.187). 

Case  HI:  ®(0)  < -(U  - d)(a  - §^3). 

According  to  Equation  (4.187),  we  derive  in  the  same  way  as  in  Equation  (C.30), 

—jum)  = (v-m  A»L(0+6)+(«+^-^} 

+ dx(0 + ~ d)a~  db),  (C.39) 

and  conclude 


sup{rf4)-,J(I(0)) : *(0)  < -{U  ~ i){a  - \rh)}  = ~ i)[a ~ \irHP- 

We  have  j^ojJf(x(0)  + (U  - d)a  - db)  < 0 for  z(0)  = -(U  - d)(a  - |^),  since 
*(0)  + (U-  d)a  - db  = -±db  < 0 for  *(0)  = —(17  — d)(a  - We  conclude 


d J^-iU-dXa-l-JL;)) 


dx(  0) 


2U-d' 


< (U  d){  2 (a  ~ 2 jj  _ ^)(a  + b)  + (a  + b)2  — jj—^2} 


U -d 


= (U-d){(a  + b)(a  + b-2(a-±v^)) 

= (£/- d){(a + &)(-«  + 6+ ^)-^62} 

Ub 2 


62} 


/rr  ,x  / 9 ,9  dab  db 

([/  - d){—a2  + b2  + — - + 77— -) 


£7-d  E7-<T  U-d 


} 
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, »\  r 9 dab  , 

= (C-<(){a(^-«)} 

< o.  (C.40) 


We  conclude 


-./2*(z(0))  < 0,  for  *(0)  < -(U  - d)(a  - \j^)- 


dx(  0) 

Case  H2:  x(0)  G (-{U  - d)(a  - 

According  to  Equation  (4.187),  we  derive  in  the  same  way  as  in  Equation  (C.31) 


d: r(0) 


J2*(x(0)) 


*2(0) 

U - d' 


(C.41) 


and  conclude 


< 0.  ^ *(°)  € (-(£/  - i)(a  - 1^),0). 

Case  H3:  x(0)  G (0,d(a—  fjnrj))-  According  to  Equation  (4.187),  we  derive  in  the 
same  way  as  in  Equation  (C.32) 

ojj2*(x(°))  = ~p>  (C>42) 

and  conclude 


^0)^2(*(0))  > o,  for  x(0)  G (0,  d(a  - 

Case  H4:  x(0)  G (d(a  - da  + (U  - y/U{U  - d))b). 

Since  we  have  S2(x(0))  < for  x(0)  G (d(a  - |j^),da  + (U  - y/U(U  - d))b),  we  can 
show  in  a similiar  way  as  in  Case  G4  in  Section  C.7  that  the  derivative  of  </2(a:(0))  with 
respect  to  x(0)  during  the  interval  (d(a  - \jjz^),da  + (U  - \/U(U  - d))b)  is  positive. 

Case  H5:  x(0)  > da  + {U  - s/U(U  - d))b. 

According  to  Equation  (4.187),  we  have  for  z(0)  > da  + (TJ  - y/U(U  - d)b, 
d 

dx(  0) 


J2*(x(0)) 
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> 


> 


£+b  2(1(0)  - dt)dt  + J*(x(0)  - d{a  + b )) 

2a:(0)(a  + b)  — d(a  + b)2  + — ^(°  + b)) 

2 {da  + ({/  - sJu{U  -d))b)(a  + 6)  - d(a  + 6)2 
2(da  + (t/  - y/u{U -d))b){a  + b)  - d(a  + b )2 
2da2  + 2dab  + 2(17  — ^(t/  - d))ab 

+2(U  - y/u{U^d))b2  - da 2 - 2dah  - db2 
da2  + 2(U  - \Ju{U  — d))ab  + 2(U  - yJu{U  - d))b2  - db2, 
da 2 + 2(17  - \JlJ{U  — d))ab  + ( Ub 2 - 2^/t7([/  - d)b2  + {U  - d)b2), 
da2  + 2{U  - yJu(U  ~d))ab  + (VU  - VW~^d)2b2, 

0. 


(C.43) 


(C.44) 


Finally,  we  observe 


lim 


d 


r(o)— >o,  x(o)>o  dx( 0) 


J2*(z(0)) 


c(0)- 


to  T^T 
-+o,  x(o)<o  aa;(0) 


^*(*(0))  = o. 


In  summary,  we  have  validated  Equation  (4.188): 


d^0)J2(x o)  < 0,  for  x(0)  < 0, 

4o)J2(^o)  = 0,  for  *(0)  = 0, 


4oj  J&o)  > o,  for  *(0)  > 0. 

C.9  An  Initial  Production  Control  for  Problem  F? 


This  is  a modification  of  the  Case  Case  A3-2  for  Problem  P2.  It  also  applies  to  Problem 
P/v  for  N > 3.  In  this  case,  we  have  x(0)  = 0,  u(t)  = U for  f G [0,a),  and  d = -^U . For 
At  > 0,  we  define  a class  of  controls  u&t(t)  by 


uAt(t)  = < 


d, 


if  t e [0,  At), 


U,  if  f 6 [At,  a), 
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which  leads  to  a trajectory  with 

0,  for  t £ [0,  At], 

xAt{t)  — (U  — d)(t  - At),  fort  6 [At,  a], 

(U  — d)(a  — At)  - d(t  — a),  for  < G [a,  a + 6], 

Notice  that  x(t)  is  identical  to  xAt(t)  for  At  = 0.  Since  we  have  x(a)  = (U  - d)a  = db 
and  a;  At  (a)  = db—(U  — d)At  < db  for  At  > 0,  we  cannot  apply  Property  (4.118)  for  Problem 
P2  • We  directly  use  the  definition  of  the  performance  measure  and  obtain 

■h(uAt)  = r {{U  - d)(t  - At)}2dt  + [a+b{(U  - d)(a  - At)  - d(t  - a))2dt 

J A t J a 

+ Jf((U  - d)(a  - At)  - db) 

ra—At  rb 

= / {(U  - d)t}2dt  + / {(U  - d)(a  — At)  - dt}2dt 

Jo  Jo 

+r1(-(u-d)At).  (c.45) 


We  take  the  derivative  with  respect  to  At  and  obtain  for  At  £ (0,a  - CT), 

J^tHu At)  = (—1){(U  — d)(a  - At)}2  + Jb  —2(U  — d){(U  — d)(a  — At)  — dt}dt 

= —(U  — d)2(a  -At)2  - 2 (U  - d)2(a  - A t)b  + (U  - d)db 2 
(-(U-d)At)\ 


~(U  - d)(- 


U -d 


= —{U  — d)2{(a  - At)2  + 2(a  — At)b  — 


U-d 


b2}  - (U  - d)2(At)2} 


—(U  — d)2{(a  - At  + b)2 


U-d 


b2  + (At)2}. 


(C.46) 


From  d = U,  we  obtain  b = Replacing  b in  Equation  (C.46)  results  in 

d J2(«At)  = ~(U  - d)2{(a  - At  + b)2  - j-^—b2  + (At)2} 

U — a 


dAt 


-( V - <i)2{(f « - At f - JL-^'LAay  + (Af)2} 


~(U  ~ df&a  - ± Atf- 


U(U  - d) 


d2  v~  U 

l 

d2  ~ 


d2 


{a}2}  + (At)2 


= ~{U  - d)2{(~  ~ U{UJ0_  d))a2  - 2^aAt  + (At)2  + (At)2} 


d2 
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-( U - dfi^a2  - 2^aA  t + (At)2  + (A/)2} 
-(U  - d)2{^a(a  - 2A t)  + 2(A t)2} 


That  means  that  for  At  G (0,min(f,a-CV)),  we  have  ^J2(uAt)  < 0 and  consequently, 
the  policy  u(t)  is  not  optimal. 

For  Problem  P/v  (TV  > 3),  the  term  J^(—(U  - d)At)  in  Equation  (C.45)  is  replaced 
by  - d)At).  Since  the  derivative  of  - d)At)  with  respect  to  At  is 

—(U  — d)2(At)2  for  sufficiently  small  At  > 0 and  for  all  N > 3,  we  will  have  the  same 
expression  for  j&J(uAt)  in  Equation  (C.46).  Thus,  the  Case  A3-2  is  validated  in  the  same 
way  for  all  TV  > 2. 

C.10  Rewriting  a quadratic  equation  (Case  C4-2) 

According  to  Equation  (4.172),  we  have 


Our  goal  in  this  section  is  to  simplify  this  quadratic  equation  and  solve  for  its  roots.  We 
rewrite  this  equation  as  follows: 


p™  = u(v 


^(d(a+b-)-x(  0))} 


^(*(0  ) + US-d(a  + b))2. 


(C.47) 


-jj—2(x{0)  + US-d(a  + b)y 

= U(U  + d)S2  + 2U(x(0)  - da  + Ub)S  + 2£76(*(0)  - d(a  + h-)) 


-jp—j(U2S2  + 2(*(0)  - d(a  + b))US  + (x(0)  - d(a  + b ))2) 


+2U(x(0)  - d(a  + b-))b  - jj^(x( 0)  - d(a  + b))2. 


(C.48) 


We  now  rewrite  the  coefficients.  The  coefficient  of  S 2 satisfies 


U3  U(U 2 - d2)  - U3  Ud 2 

U -d~  U-d 


U(U  + d)~ 


U -d' 


(C.49) 
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the  coefficient  of  S1  = S satisfies 


, u2 

2(U(x(0)  - da  + JJb)  - — — -(a:(0)  _ d(a  + &))) 

nU(x(0)  -da+  Ub)(U  - d)  - U2(x( 0)  - d(a  + b)) 

U-d 

oC/2(x(0)  - da  + Ub)  - Cd(z(0)  - da  + t/6)  - U2(x( 0)  - d(a  + b )) 

U-d 

nU2(Ub)~  Ud(x(0)-da) 

U-d 

= f^2(f/26-d(x(0)-rfa)), 


and  the  coefficient  of  5°  (the  constant  in  Equation  (C.48))  satisfies 

2U{x(0)  - d(a  + ^ ))b  - 0)  - d(a  + b ))2 

= 2Ubx(0 ) - 2U dab  - Udb2  - — — --^(x2(0)  - 2ar(0)d(a  + 6)  + d2(a  + 6)2) 
= + *(0){2(ttt+  ^d(a  + &))} 

+ {-2Udab  - Udb 2 - — -d2(a  + b )2}. 


We  simphfy  Equation  (C.51)  further  by  rewriting  the  coefficients  of  x(0)  and  x° 
have 


{2  (Pi  + UUd(a  + (,))}  = {2P(t'-^+M(°  + ^)}  = 


U-d 


U-d 


U 

U-d 


2 (Ub  + da), 


and 


{-2Cda6  - Udb 2 - j^——-^d2{a  + b)2} 

f {—2Udab  - Udb2){U  - d)  - Ud2(a  + 6)2 , 

1 U-d  * 

= {jj^-j{(-2dab  - db2)(U  - d)  - d2(a  + b )2}} 

= {-^-—^{—2U dab  - Udb 2 + 2d2ab  + d2b 2 - d2a2  - 2 d2ab  - d2b2}} 
= {^—^{-2Udab-  Udb2-d2a2}} 

= {-jj^dida2 + 2Uab  + Ub2}}. 

We  now  substitute  Equations  (C.52)  and  (C.53)  into  Equation  (C.51)  and  obtain 

x2(°H-  iTT^  + *(°){2 (Ub+  jj^dia  + 6))} 


(C.50) 


(C.51) 
i).  We 


(C.52) 


(C.53) 
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+{-2 Udab  - Udb2  - d2(a  + b )2} 

= -jj—^x2(0)  + — ■ - j2(Ub  + da)x(O)  - ^ ^ d{da 2 + 2Uab  + C/62} 

= -^^{x2(0)-2(t/6  + da)x(0)  + d(da2  + 2Ca&+f/h2)}.  (C.54) 

We  now  substitute  the  results  of  Equations  (C.49),  (C.50),  and  (C.54)  into  Equation  (C.48) 
and  obtain 

Ts' h{s) 

= (17(17  + d)  - j^)S2  + 2(I7(*(0)  - da  + Ub ) - ^-^(x(0)  - d(a  + 6)))5 
+2C(x(0)  - d(a  + h-))b  - ^-^(x(0)  - d(a  + 6))2 
= -^52  + 2^(C/26-d(x(0)-da))5 

-^-^{x2( 0)  - 2(Ub  + da)x(0)  + d{da 2 + 2Uab  + C62)} 

^ fo2  0C26  - d(z(0)  - da)  r 

U - d1  d2  * 

, x2(0)-2(C&4-da)x(0)  + d(da2  + 2Ca6+C&2)1 
+ }.  (C.55) 

For  simplification,  we  introduce  f(x( 0)),  Xj?  and  x2  as  follows: 


/(*( °))  = 

x2(0)  - 2(Ub  + da)x(0)  + d(da2  + 2Ca6  + C/62) 

(C.56) 

d2 

®2  = 

C/6  + da  - yJu(U  - d)b  = da  + b(U  - y/u(U  - d)), 

(C.57) 

A = 

Ub  da  + \ju(U  — d)b. 

(C.58) 

We  now  consider  the  function  f(x( 0)).  We  have 

/(x(0))  — — (C6  + da))2  — (C/6  + da)2  + d(da2  + 2Uab  + Ub2) 

d2 

Since  we  have 

(Ub  + da)2  - d(da2  + 2Uab  + Ub2)  = U2b2  + 2U dab  + d2a2  - d2a2  - 2U dab  - Udb2 

= U(U  — d)b2, 

we  conclude 


/(*(0))  = 0 


if  and  only  if 


X(0) £ {*2,*2}> 


(C.59) 
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and 


/(*( 0))  < 0 


if  and  only  if  z(0)  6 (x2,  x2). 


(C.60) 


We  observe 


(da  + b(U  - \Ju(U  - d )),  d(a  + b))  = (x2,d(a  + b))  C (x2,  x").  (C.61) 

From  Equations  (C.59),  (C.60)  and  (C.61),  we  conclude 

/(*( 0))  < 0,  for  z(0)  G (da  + b(U  - y/u{U  - d)),d(a  + b)),  (C.62) 

f(x(0))  > 0,  for  ar(0)  G (d(a  - da  + b(U  - yJu(U  - d))).  (C.63) 


We  further  define 


, U2b-d(x(0)-da)  'l,U*b-d(x(0)-da)„  „ ,nS, 

fp.  \/v  ^2  ) /(x(0))j 


d2 


S'o 


U2b  - d(x( 0)  - da)  U2b  - d(x(0)  - da) , 


2 - J2 + \/(  " "X d ""y)2~/(a:(Q))- 


According  to  Equation  (C.55),  we  have  for  the  second  derivative 

d2  T,c,  „ M2  (c  U2b-d(x(0)-da), 

ds^MS)  = ~2—d{s ■>' 

We  evaluate  Equation  (C.66)  at  S = S2  and  S = S"  and  obtain 

^MS'2)  > o. 


(C.64) 

(C.65) 

(C.66) 


dS 2 


Wl)  < 0. 


Since  our  objective  is  to  minimize,  we  focus  on  S2.  We  rename  S2  as  52(z(0))  and  have 

^ - /(*(<>)).  (C.67) 


S2(z(0))  = 


U2b  - d(x(0)  — da)  I U2b  — d(x( 0)  - da) 


d2 


(' 


d2 


According  to  Equation  (C.67),  we  conclude 


S2(z(0))  < 0,  if  and  only  if  f(x( 0))  < 0. 

We  now  use  Equations  (C.62)  and  (C.68)  to  obtain 

S2(*(0))  < 0,  for  ar(O)  € (da  + b(U  - y/U(U  - d)),  d(a  + &)), 
S2(z(0))  > 0,  for  z(0)  € (d(a  - da  + b(U  - y/U(U  - d))). 


(C.68) 


(C.69) 
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C.ll  ff^adOV)  is  monotone  decreasing  (C4-2-2) 

In  this  section,  we  show  that  the  function  ^(^(O))  is  monotone  decreasing  for  x(0)  £ 
[d(a  - CT),d(a+  |)],  i.e., 


dx(  0) 


52(x(0))  < 0,  for  x(0)  6 (d(a  - Cr),d(a  + -)). 


(C.70) 


According  to  Equation  (4.174),  we  obtain  the  following  expression  for  the  derivative  of 
S2(z(0))  with  respect  to  x(0): 


d 


dx(  0) 


52(x(0)) 


d 2^U_ib-d(,(0)-da1)2  _ /(a,(Q)) 
d :/(*( 0))}> 


{2( 


U2b  — d(x(0)  — da)  1 


d 2 


da:(0)' 


We  now  evaluate  the  term  {2(— b-dty°'>  da))(-^)  - ^/(a;(0))}: 

= ii(g*-^o)-*)x_i)_  < wo)) 


d2 


d dx( 0)' 


n/U2b-d(x(Q)-da)^  1^  2x(0)-2  (Ub  + da) 

( d2  ){  d’  d2 

2 U 2 

= -^(-^-6  - (a:(0)  - da)  + x(0)  - (f/6  + da)) 

2 t/2 

= 

Substituting  Equation  (C.72)  into  Equation  (C.71),  we  obtain 

d 


da:(0) 

1 


S2(z(0)) 


1 2 U2 

~d  ~ - /(x(0))(~^>(t  ” U)i 


1 

~d  + 


^2\/( 


U2b—d(x(0)—da) 


d? 


y - /(x(o» 


(£  _ [/)>-  _ /(t(0)) 


<;2d( 


U2b—d(x(0)—da)  \2 


d2 


)2  - /(*(0)) 


^d) 


(C.71) 


(C.72) 


(C.73) 


dar(0) 


S2(z(0))  < o 


Thus,  we  have 


196 


if  and  only  if 


(—  _ ufb 2 _ d2^u2b-  dW)~  da)^ 


- fwm  < o. 


(C.74) 


According  to  Equation  (C.56),  we  now  evaluate  the  expression  (U  b d'U  f-J(x(0)): 

U2b-d(x(0)-da)  2 
v ^2  ' /(x(0)) 

U2b  - d(x( 0)  - da)  2 z2(0)  - 2 (Ub  + da)*(0)  + d(da2  + 2Uab  + Ub 2) 

( d2  j d 2 

1 d2 

= ~ (*(0)  - da))2  - x2(0)  + 2{Ub  + da)ar(O)  - d(da2  + 2£7af>  + 1762)) 

1 U4  U 2 

= ^(-^pb2  - 2— 6(x(0)  — da)  + (x(0)  — da)2 

~(x{0)  - da)2  + 2(Ub)x(0)  - d(2Uab  + Ub2)) 

1 U4  U 2 

= -p  (-^-b2  - 2— 6(z(0)  - da)  + 2(Ub)x(0)  - d(2Uab  + Ub2)) 

= — U d)b2  — 2(-^-b  — Ub)(x(0)  — da))  (C.75) 


Substituting  Equation  (C.75)  in  Equation  (C.74),  we  obtain 

U2b  — d{x{  0)  — da) 


U2 

d 

U2 


c-  - ufb2  - d2{(— - >f  - f(x( 0))} 


= (^-  - ufb 2 - ((—  - t/d)62  - 2(^6  - Ub)(x(0)  - da)) 
2(^6  - £76)(x(0)  - da)  - ((^J  - Cd)62  - (^  - C)262) 
2(l-jb  - Ub)(x( 0)  - da)  - (( J - t/d)  - (^  - Uf)b2. 


(C.76) 


We  rewrite  (^-6  - t/6)  and  obtain 


,f/2,  f72-Cldt  £7-drri 

(T6  _ (,6)  =__6 


(C.77) 


For  {(Up  — Ud)  — (^  — C)2),  we  obtain 


r/2 


U 3 

= -Ud  + 2^-  -U2 

d 


= U 


U 


2U2  -d2  -Ud 
d 

U2-d2  + U2-  Ud 
d 
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(U  - d)(U  + d)  + (U  - d)U 
d 

= l-^U((U  + d)  + U).  (C.78) 

Substituting  Equations  (C.77)  and  (C.78)  into  Equation  (C.76),  we  obtain 

2^—^-—Ub(x(0)  - da)  - + d)  + U)b2 

= 2(^-Ub)((x(0)-da)-^((U  + d)  + U)b) 

= 2(^^Ub)(x(0)-(Ub  + d(a  + ^))). 

We  thus  conclude 


^^52(a;(0))  < 0,  for  ar(0)  < Ub  + d(a  + ^), 

which  validates  Equation  (C.70).  Thus,  the  maximum  value  of  S2(x(0))  during  the  interval 
[d(a  - | TfZd),  d(a  + |)]  is  obtained  at  x(0)  = a - \ 

C.12  Evaluation  of  So(d(a  - ^p^j) ) (Case  C4-2-21 

In  this  section,  we  show 


S2(d(a 


1 db  1 db 

2 U — d ~ a ~ 2U  — d 


(C.79) 


According  to  Equation  (4.174),  S2(z(0))  is  given  by 


S2(z(0)) 


U2b  - d(x(0)  - da)  U2b  - d(x( 0)  - da) 


d 2 


-\  (- 


d 2 


[)2-f(x(  0)).  (C.80) 


where  /(*( 0))  = £!Mz?(^H.iaMo)+rf(^+2^a6+^)  ^ Wg  firgt  rewrite  /(x(0)).  We  have 


f(x(0))  = ^{x2(0)-2(Ub  + da)x(0)  + d2a2  + 2Udab+Udb2} 

= ^{(z(0)  - (Ub  + da))2  - ( U2b 2 + 2Udab  + d2a2)  + d2a 2 + 2Udab  + Udb2} 
= ^{(*(0)  -(Ub  + da))2  - U(U  - d)b2}. 


We  now  evaluate  the  function  f(x( 0))  for  x(0)  = d(a  - \ jjZd)'- 


f(d(a  - 


1 db 

2 U -d’ 
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= ~ ( Ub  + da))2  - U(U  - d)b2} 

= w^p{{i^u(v-d)f-lJ{u-if} 

= {V-  d)V{ T + “ d)d* + ^ " rf)2  “ ,J(,J  - rf>3* 

= (If  ^d)V{J  + v(u~  d)d 2 + ^ - <f)2<i} 

= (rap4+!,2(c'-‘,»- 

We  proceed  to  evaluate  the  expression  (^.2&~^0)~rfa))2  _ /(x(0))  under  the  squareroot  (see 


Equation  (C.80))  for  a;(0)  = d(a  - ^^j): 


( 


U2b  — d(x(  0)  — da) 
d 2 

U2b  - d(d(a  - \j£^)  - da) 


f ~ Ma 


1 db 


2 U — d 


)) 


d2 


)2- 


b2 


= ( 


f/26  + d21-^- 

o U -I-  a 2 

V d2  j 

U2b  1 db 


( U - d)2d2  1 4 


{“T  + U2(U  — d)d) 


b2 


(U  - d)2d2  1 4 


{^  + f/2(d-d)d} 


d2 


+ x 


?)) 


b2 


2U-d"  (U  — d)2d2  1 4 


{^  + t/2(t/-d)d} 


d4 


62  + ^6 


db 


+ 


1 d2b2 


b 2 


d2  U-d  4{U-d)2  (U  — d)2d2 1 4 


{^  + tf2(C/-d)d} 


62 


n4(U  - dl2  d4  d4 

(t/ - d)2d2~{  d2  + ^ ~ ^ + T - (T  + ^ 


62 


(U  -d)2d2 
b2  rU4 


d2 

r U\U-d)2 
X d2  } 


d2*d2^ 

■ 4‘>2- 


(C.81) 


Therefore,  we  have  according  to  Equations  (C.80)  and  (C.81) 

U2b  - d(d{a  - \-^)  - da) 


*«-5i£i» 


d2 


-{( 


t/26-d(d(a-ij#^-da)) 


d2 


')2 


-/(d(a-  i-^-))}5 
M v 2 U-d,)S 

U2b  - d(d(a  - Ij^)  - da) 

d2 
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1 db 

2 U -d’ 


which  vahdates  Equation  (C.80). 


(C.82) 


REFERENCES 


[1]  Abdul-Razaq,  T.  and  C.  Potts.  “Dynamic  Programming  State-Space  Relaxation 
for  Single-Machine  Scheduling,”  Journal  of  the  Operational  Research  Society , 39,  pp. 
141-152,  1988. 

[2]  Ahmadi,  R.  and  U.  Bagchi.  “Single  Machine  Scheduling  to  Minimize  Eqaliness  Subject 
to  Deadlines,”  Working  Paper  86/86-4-17,  Department  of  Management,  University  of 
Texas,  Austin,  1986. 

[3]  Ahmadi,  R.  and  U.  Bagchi.  “Just-in-Time  Scheduling  in  Single  Machine  Systems,” 
Working  Paper  85/86-4-21,  Department  of  Management,  University  of  Texas,  Austin, 
1986. 

[4]  Akella,  R.  A.,  Y.  F.  Choong,  and  S.  B.  Gershwin.  “Performance  of  Hierarchi- 
cal Production  Scheduling  Policy,”  IEEE  Transactions  on  Components,  Hybrids, 
and  Manufacturing  Technology,  CHMT-7:3,  pp.  225-240,  1984. 

[5]  Akella,  R.  A.  and  P.  R.  Kumar.  “Optimal  Control  of  Production  Rate  in  a Failure 
Prone  Manufacturing  System,”  IEEE  Transactions  on  Automatic  Control . AC-31:2, 
pp.  116-126,  1986. 

[6]  Anger,  F.D.,  C.-Y.  Lee,  and  L.  A.  Martin-Vega,  “Single  Machine  Scheduling  with 
Tight  Windows,”  Research  Report  86-16,  Department  of  Industrial  and  Systems 
Engineering,  University  of  Florida,  1986. 

[7]  Bagchi,  U.  “Scheduling  to  Minimize  Earliness  and  Tardiness  Penalties  With  a Com- 
mon Due  Date.”  Working  Paper,  Department  of  Management,  University  of  Texas, 
Austin,  1985. 

[8]  Bagchi,  U.  “Due  Date  or  Deadline  Assignment  to  Multi-Job  Customer  Orders,” 
Working  Paper,  Department  of  Management,  University  of  Texas,  Austin,  1989. 

[9]  Bagchi,  U.,  Y.  Chang  and  R.  Sullivan.  “Minimizing  Absolute  and  Squared  Deviations 
of  Completion  Times  with  Different  Earliness  and  Tardiness  Penalties  and  a Common 
Due  Date,”  Naval  Research  Logistics  Quarterly,  34,  pp.  739-751,  1987. 

[10]  Bagchi,  U.,  R.  Sullivan,  and  Y.  Chang.  “Minimizing  Mean  Absolute  Deviation  of 
Completion  Times  About  a Common  Due  Date,”  Naval  Research  Logistics  Quarterly, 
33,  pp.  227-240,  1986. 

[11]  Bagchi,  U.,  R.  Sullivan,  and  Y.  Chang.  “Minimizing  Mean  Squared  Deviation  of 
Completion  Times  About  a Common  Due  Date,”  Management  Science,  33,  pp.  894- 
906,  1987. 

[12]  Bai,  S.  X.  and  J.  H.  Burlianpurwala.  “Hierarchical  Production  Control  for  a Manu- 
facturing System  with  Dynamic  Setup  Changes,”  Research  Report  94-2,  Department 
of  Industrial  and  Systems  Engineering,  Gainesville,  1994. 


200 


201 


[13]  Bai,  S.  X.  and  M.  Elhafsi.  “Real-Time  Scheduling  of  a Manufacturing  System  with 
non-resumable  Setup  Changes,”  Research  Report  93-28,  Department  of  Industrial 
and  Systems  Engineering,  Gainesville,  1993. 

[14]  Bai,  S.  X.  and  S.  B.  Gershwin.  “Scheduling  Manufacturing  Systems  with 

Work-In-Process  Inventory  Control:  Single- Part-Type  Systems,”  To  appear  in 

I IE  Transactions.  1994. 

[15]  Bai,  S.  X.  and  S.  B.  Gershwin.  “Scheduling  Manufacturing  Systems  with  Work- 

In-Process  Inventory  Control:  Multiple-Part-Type  Systems,” 

International  Journal  of  Production  Research , Vol.  32,  No.  2,  pp.  365-385,  1994. 

[16]  Bai,  S.  X.  and  S.  B.  Gershwin.  “Scheduling  Manufacturing  Systems  with  Work- 
In-Process  Inventory  Control:  Reentrant  Systems,”  To  appear  in  HE  Transactions . 
1994. 

[17]  Baker,  K.  R.  “Introduction  to  Sequencing  and  Scheduling,”  John  Wiley  and  Sons, 
New  York,  1974. 

[18]  Baker,  K.  R.  and  A.  Chadowitz.  “Algorithms  for  Minimizing  Earliness  and  Tardiness 
Penalties  With  a Common  Due  Date,”  Working  Paper  No.  240,  Amos  Tuck  School 
of  Business  Administration,  Dartmouth  College,  Hanover,  N.  H.,  1989. 

[19]  Baker,  K.  R.  and  G.  D.  Scudder.  “On  the  Assignment  of  Optimal  Due  Dates,” 
Journal  of  the  Operational  Research  Society , 40,  pp.  93-95,  1989. 

[20]  Baker,  K.  R.  and  G.  D.  Scudder.  “Sequencing  with  Earliness  and  Tardiness  Penalties: 
A Review,”  Operations  Research,  38:1,  pp.  22-36,  1990. 

[21]  Bector,  C.,  Y.  Gupta,  and  M.  Gupta.  “Determination  of  an  Optimal 
Common  Due  Date  and  Optimal  Sequence  in  a Single  Machine  Job  Shop,” 
International  Journal  of  Production  Research , 26,  pp.  613-628,  1988. 

[22]  Bryson,  A.  E.,  Jr.  and  Y.-C.  Ho.  “Applied  Optimal  Control:  Optimization,  Estima- 
tion, and  Control,”  Blaisdell  Publishing,  Waltham,  MA,  1969. 

[23]  Chand,  S.  and  H.  Schneeberger.  “Single  Machine  Scheduling  to  Minimize  Weighted 
Earliness  Subject  to  No  Tardy  Jobs,”  European  Journal  of  Operational  Research , 34, 
pp.  221-2.30,  1988. 

[24]  Cheng,  T.  C.  E.  “Optimal  Due  Date  Determination  and  Sequencing  of  n Jobs  on  a 
Single  Machine,”  Journal  of  the  Operational  Research  Society , 35,  pp.  433-437,  1984. 

[25]  Cheng,  T.  C.  E.  “An  Algorithm  for  the  CON  Due  Date  Determination  and  Sequencing 
Problem.”  Computers  and  Operations  Research , 14,  pp. 537-542,  1987. 

[26]  Cheng,  T.  C.  E.  “Optimal  Common  Due  Date  with  Limited  Completion  Time  Devi- 
ation,” Computers  and  Operations  Research , 15,  pp.  91-96,  1988. 

[27]  Cheng,  T.  C.  E.  and  M.  Gupta.  “Survey  of  Scheduling  Research  Involving  Due  Date 

Determination  Decisions,”  European  Journal  of  Operational  Research , 38,  pp.  156- 
166,  1989.  ~~ 

[28]  Cook,  S.  A.  “The  complexity  of  theorem-proving  procedures,”  Proceedings  of  the 
3rd  Annual  ACM  Symposium  on  the  Theory  of  Computing,  pp.  151-158,  1971. 

[29]  Coffman,  E.  G.,  Jr.  (ed.).  “Computer  and  Job  Shop  Scheduling  Theory,”  Wiley,  New 
York,  1976. 


202 


[30]  Conway,  R.  W.,  W.  L.  Maxwell,  and  L.  W.  Miller.  “Theory  of  Scheduling,”  Addison- 
Wesley,  Reading,  MA,  1967. 

[31]  Davis,  J.  and  J.  Kanet.  “Single  Machine  Scheduling  With  a Nonregular  Convex 
Performance  Measure,”  Working  Paper,  Department  of  Management,  Clemson  Uni- 
versity, Clemson,  S.C.,  1988. 

[32]  De,  P.,  J.  Ghosh,  and  C.  Wells.  “A  Note  on  the  Minimization  of  Mean  Squared 
Deviation  of  Completion  Times  About  a Common  Due  Date.”  Management  Science , 
35,  pp.  1143-1147,  1989. 

[33]  De,  P.,  J.  Ghosh,  and  C.  Wells.  “Scheduling  About  a Common  Due  Date  With 
Earliness  and  Tardiness  Penalties,”  Working  Paper,  University  of  Dayton,  Dayton 
OH,  1989. 

[34]  Dempster,  M.  A.  H.,  J.  K.  Lenstra,  and  A.  H.  G.  Rinnooy  Kan  (eds.).  “Deterministic 
and  Stochastic  Scheduling,”  Vol.  1,  Reidel,  Dordrecht,  Netherlands,  1982. 

[35]  Eilon,  S.  and  I.  Chowdhury.  “Minimizing  Waiting  Time  Variance  in  the  Single  Ma- 
chine Problem,”  Management  Science , 23,  pp.  567-575,  1977. 

[36]  Elsayed,  E.  A.  and  C.  C.  Hwang.  “Analysis  of  Two  Stage  Manufacturing 
Systems  with  Buffer  Storage  and  Redundant  Machines,”  International  Journal 
of  Production  Research , 24,  pp.  187,  1986. 

[37]  Emmons,  H.  “One-machine  sequencing  to  minimize  certain  functions  of  job  tardi- 
ness,” Operations  Research , 17,  pp.  701-715,  1969. 

[38]  Emmons,  H.  “Scheduling  to  a Common  Due  Date  on  Parallel  Uniform  Processors,” 
Naval  Research  Logistics  Quarterly , 34,  803-810,  1987. 

[39]  Federgruen,  A.  and  G.  Mosheiov.  “Efficient  Algorithms  for  Scheduling  Problems  with 
General  Earliness  and  Tardiness  Cost  Structures,”  Working  Paper,  Graduate  School 
of  Business,  Columbia  University,  1991. 

[40]  French,  S.  “Sequencing  and  Scheduling:  An  Introduction  to  the  Mathematics  of  the 
Job-Shop,”  John  Wiley  and  Sons,  1982. 

[41]  Fry,  T.  and  G.  Leong.  “A  Bi-Criterion  Approach  to  Minimizing  Inven- 
tory Costs  on  a Single  Machine  When  Early  Shipments  are  Forbidden,” 
Computers  and  Operations  Research , 14,  pp.  363-368,  1987. 

[42]  Fry,  T.,  R.  Armstrong,  and  J.  Blackstone.  “Minimizing  Weighted  Absolute  Deviation 
in  Single  Machine  Scheduling,”  IEEE  Transactions.  19,  pp.  445-450,  1987. 

[43]  Fry,  T.,  K.  Darby-Dowman,  and  R.  Armstrong.  “Single  Machine  Scheduling  to  Mini- 
mize Mean  Absolute  Lateness,”  Working  Paper,  College  of  Business  Administration, 
University  of  South  Caroline,  CO,  1988. 

[44]  Fry,  T.,  G.  Leong,  and  T.  Rakes.  “Single  Machine  Scheduling:  A Comparison  of  Two 
Solution  Procedures,”  OMEGA  15,  pp.  277-282,  1987. 

[45]  Garey,  M.  R.  and  D.  S.  Johnson.  “Strong  NP-completeness  results:  motivation, 
examples  and  implications,”  Journal  of  the  Association  of  Computing  Machines , 25, 
pp.  499-508,  1978. 

[46]  Garey,  M.  R.  and  D.  S.  Johnson.  “Computers  and  Intractibility:  A Guide  to  the 
Theory  of  NP-completeness,”  Freeman,  San  Francisco,  1979. 


203 


[47]  Garey,  M.  R.,  R.  Tarjan,  and  G.  Wilfong.  “One-Processor  Scheduling  With  Symmetric 
Earliness  and  Tardiness  Penalties,”  Mathematics  and  Operations  Research,  13,  pp. 
330-348,  1988. 

[48]  Gershwin,  S.  B.  “Hierarchical  flow  control:  a framework  for  scheduling  and 

planning  discrete  events  in  manufacturing  systems,”  Proceedings  of  the  IEEE, 
Special  Issue  on  Dynamics  of  Discrete  Event  Systems , 77:1,  pp.  195-209,  1989. 

[49]  Gershwin,  S.  B.,  R.  A.  Akella,  and  Y.  F.  Choong.  “Short-term  production  scheduling 
of  an  automated  manufacturing  facility.”  IBM  Journal  of  Research  and  Development, 
29-4,  pp.  392-400,  1985. 

[50]  Graham,  R.  L.,  E.  L.  Lawler,  J.  K.  Lenstra,  and  A.  H.  G.  Rinnooy  Kan.  “Opti- 
mization and  approximation  in  deterministic  sequencing  and  scheduling:  a survey,” 
Annals  of  Discrete  Mathematics,  5,  pp.  287-326,  1979. 

[51]  Graves,  S.  C.  “A  review  of  production  scheduling,”  Operations  Research,  26,  pp. 
200-203,  1978. 

[52]  Gupta,  S.  and  J.  Kyparisis.  “Single  Machine  Scheduling  Research,”  OMEGA . 15,  pp. 
207-227,  1987. 

[53]  Gupta,  S.  and  T.  Sen.  “Minimizing  a Quadratic  Function  of  Job  Lateness  on  a Single 
Machine,”  Engineering  Costs  and  Production  Economist , 7,  pp.  181-194,  1983. 

[54]  Hall,  N.  G.  “Single  and  Multi-Processor  Models  for  Minimizing  Completion  Time 
Variance,”  Naval  Research  Logistics  Quarterly , 33,  pp.  49-54,  1986. 

[55]  Hall,  N.  G.  and  M.  E.  Posner.  “Earliness- Tardiness  Scheduling  Problems,  I:  Weighted 
Deviation  of  Completion  Times  about  a Common  Due  Date,”  Operations  Research, 
39,  pp.  836-846,  1991. 

[56]  Hall,  N.  G.,  W.  Kubiak,  and  S.  P.  Sethi.  “Earliness-Tardiness  Scheduling  Prob- 
lems, II:  Deviation  of  Completion  Times  about  a Restrictive  Common  Due  Date,” 
Operations  Research , 39,  pp.  847-856,  1991. 

[57]  Hestenes,  M.  R.  “Calculus  of  Variations  and  Optimal  Control  Theory,”  Wiley,  New 
York,  1966. 

[58]  Hoogeveen,  J.  A.  and  S.  L.  Van  De  Velde.  “Scheduling  Around  a Small  Common  Due 
Date,”  European  Journal  of  Operational  Research,  55,  pp.  237-242,  1991. 

[59]  Hopp,  W.  J.,  N.  Pati,  and  P.  C.  Jones.  “Optimal  Inventory  Control  in  a Produc- 
tion Flow  System  with  Failures,”  Technical  Report  88-07,  Department  of  Industrial 
Engineering  and  Management  Sciences,  Northwestern  University,  IL,  1988. 

[60]  Hu,  J.-Q.  and  D.  Xiang.  “Optimal  Production  Control  for  Failure  Prone  Manu- 
facturing Systems  with  Deterministic  Machine  Up  Times,”  Working  Paper,  Boston 
University,  1992. 

[61]  Ignall,  E.  and  A.  Silver.  “The  Output  of  a Two  Stage  System  with  Unreliable  Ma- 
chines and  Limited  Storage,”  A.I.I.E.  Transactions.  9,  pp.  183  ff.,  1977. 

[62]  Johnson,  D.  S.  “The  NP-completeness  column:  an  ongoing  guide,” 

Journal  of  Algorithms,  4,  pp.  189-203,  1983. 

[63]  Kanet,  J.  “Minimizing  the  Average  Deviation  of  Job  Completion  Times  About  a 
Common  Due  Date,”  Naval  Research  Logistics  Quarterly,  28,  pp.  643-651,  1981. 


204 


[64]  Karp,  R.  M.  “Reducibility  among  combinatorial  problems,”  Complexity  of  Computer 
Computations,  R.  E.  Miller  and  J.  W.  Thatcher  (eds.),  Plenum  Press,  New  York,  pp. 
85-103,  1972. 

[65]  Karp,  R.  M.  “On  the  computational  complexity  of  combinatorial  problems,” 
Networks.  5,  pp.  45-68,  1975. 

[66]  Kimemia,  J.  and  S.  B.  Gershwin.  “An  Algorithm  for  the  Computer  Control  of  a 
Flexible  Manufacturing  System,”  HE  Transactions.  15:4,  pp.  353-362,  1983. 

[67]  Kirk,  D.  E.  “Optimal  Control  Theory,  An  Introduction,”  Network  Series.  Prentice- 
Hall,  1970. 

[68]  Kramer,  F.-J.  “Essays  in  Production  Scheduling  with  Work-In-Process  related  per- 
formance measures.”  Ph.D.  Dissertation,  University  of  Florida,  Gainesville,  FL,  1994. 

[69]  Kramer,  F.-J.  and  S.  X.  Bai.  “A  Production  Flow  Control  Problem  with  Periodic 
Maintenance,”  Research  Report,  Industrial  and  Systems  Engineering,  University  of 
Florida,  Gainesville,  1994. 

[70]  Kramer,  F.-J.  and  S.  X.  Bai.  “Optimal  Control  of  a Production  System  with  Periodic 
Maintenance:  Part  I:  Solution  of  the  one-cycle  Problem,”  Working  Paper,  Industrial 
and  Systems  Engineering,  University  of  Florida,  Gainesville,  1994. 

[71]  Kramer,  F.-J.  and  S.  X.  Bai.  “Optimal  Control  of  a Production  System  with  Periodic 
Maintenance:  Part  II:  Solution  of  the  iV-cycle  Problem,”  Working  Paper,  Industrial 
and  Systems  Engineering,  University  of  Florida,  Gainesville,  1994. 

[72]  Kramer,  F.-J.  and  C.-Y.  Lee.  “Common  due  window  scheduling,”  to  appear  in 
Production  and  Operations  Management , 1994. 

[73]  Kramer,  F.-J.,  and  C.-Y.  Lee.  “Due  Window  Scheduling  for  Parallel  Machines,” 
to  appear  in:  Mathematical  and  Computer  Modelling , Special  Issue  on  Scheduling: 
Theory  and  Applications,  1994. 

[74]  Lakshminarayan,  S.,  R.  Lakshmanan,  R.  L.  Papineau,  and  R.  Rochette. 
“Optimal  Single-Machine  Scheduling  with  Earliness  and  Tardiness  Penalties,” 
Operations  Research , 26,  pp.  1079-1082,  1978. 

[75]  Lawler,  E.  L.,  J.  K.  Lenstra,  A.  H.  G.  Rinnooy  Kan,  and  D.  B.  Shmoys.  “Sequencing 
and  Scheduling:  Algorithms  and  Complexity,”  Report  BS-R8909,  Centre  for  Mathe- 
matics and  Computer  Science,  Amsterdam,  Netherlands,  1989. 

[76]  Lee,  C.-Y.  “Earliness- Tardiness  Scheduling  Problems  with  Constant  Size  of  Due-Date 
Window,”  Working  Paper  91-17,  Department  of  Industrial  and  Systems  Engineering, 
University  of  Florida,  1991. 

[77]  Lee,  C.-Y.,  S.  Danusaputro,  and  C.S.  Lin.  “Minimizing  Weighted  Number  of 
Tardy  Jobs  and  Weighted  Earliness- Tardiness  Penalties  about  a Common  Due  Date,” 
Computers  and  Operations  Research , 18,  pp.  379-389,  1991. 

[78]  Lenstra,  J.  K.  and  A.  H.  G.  Rinnooy  Kan.  “New  directions  in  scheduling  theory,” 
Operations  Research  Letters , 2,  pp.  255-259,  1984. 

[79]  Lenstra,  J.  K.,  A.  H.  G.  Rinnooy  Kan,  and  P.  Brucker.  “Complexity  of  machine 
scheduling  problems,”  Annals  of  Discrete  Mathematics , 1,  pp.  343-362,  1977. 


205 


[80]  Liman,  S.  D.  and  C.-Y.  Lee.  “Error  Bound  for  a Heuristic  on  the  Common  Due-Date 
Scheduling  Problem,”  ORSA  Journal  of  Computing,  5,  420-425,  1993. 

[81]  Lou,  S.  X.  C.,  S.  P.  Sethi,  and  Q.  Zhang.  “Optimal  Feedback  Pro- 
duction Planning  in  a Stochastic  Two-Machine  Flowshop,”  to  appear  in 
European  Journal  of  Operational  Research , Special  Issue  on  Stochastic  Control  The- 
ory and  Operational  Research,  1994. 

[82]  Malathronas,  J.  P.,  J.  D.  Perkins,  and  R.  L.  Smith.  “The  Availability  of  a Sys- 
tem of  Two  Unreliable  Machines  Connected  by  an  Intermediate  Storage  Tank,” 
A.I.I.E.  Transactions.  15,  pp.  195  ff.,  1983. 

[83]  Nahmias,  S.  “Production  and  Operations  Analysis,”  Irwin  Inc.,  Boston,  MA,  1989. 

[84]  Ow,  P.  and  T.  Morton.  “The  Single  Machine  Early-Tardy  Problem,”  Management 
Science . 35,  pp.  177-191,  1989. 

[85]  Panwalkar,  S.,  M.  Smith,  and  A.  Seidmann.  “Common  Due  Date  Assignment  to  Min- 
imize Total  Penalty  for  the  One  Machine  Scheduling  Problem,”  Operations  Research , 
30,  pp.  391-399,  1982. 

[86]  Pontryagin,  L.  S.,  V.  G.  Boltyanskii,  R.  V.  Gamkrelidze,  and  E.  F.  Mishchenko.  “L.  S. 
Pontryagin  Selected  Works,”  Vol.  4.  The  Mathematical  Theory  of  Optimal  Processes. 
Gordan  and  Breach  Science  Publishers,  Montreux,  Switzerland,  1986. 

[87]  Presman,  E.,  S.  P.  Sethi,  and  Q.  Zhang.  “Optimal  Feedback  Productin 
Planning  in  a Stochastic  N-Machine  Flowshop,”  to  appear  in  Proceedings 
of  the  12th  World  Congress  of  International  Federation  of  Automatic  Control , Syd- 
ney, Australia,  1993. 

[88]  Quaddus,  M.  “A  Generalized  Model  of  Optimal  Due-Date  Assignment  by  Linear 
Programming,”  Journal  of  the  Operational  Research  Society,  38,  pp.  353-359,  1987. 

[89]  Raghavachari,  M.  “A  V-Shape  Property  of  Optimal  Schedule  of  Jobs  About  a Com- 
mon Due  Date,”  European  Journal  of  Operational  Research , 23,  pp.  401-402,  1986. 

[90]  Raghavachari,  M.  “Scheduling  Problems  With  Non-Regular  Penalty  Functions  - A 
Review,”  Opsearch , 25,  pp.  141-164,  1988. 

[91]  Rinnooy  Kan,  A.  H.  G.  “Machine  Scheduling  Problems:  Classification,  Complexity 
and  Computations,”  Nijhoff,  The  Hague,  Netherlands,  1976. 

[92]  R.yzin,  G.  van,  S.  X.  C.  Lou,  and  S.  B.  Gershwin.  “Production  control  for  a two- 
machine  system,”  to  appear  in  HE  Transactions.  1994. 

[93]  Seidmann,  A.,  S.  Panwalkar,  and  M.  Smith.  “Optimal  Assignment  of  Due-Dates  for  a 
Single  Processor  Scheduling  Problem,”  International  Journal  of  Production  Research. 
19,  pp.  393-399,  1981. 

[94]  Sen,  T.  and  S.  Gupta.  “A  State-of-the-Art  Survey  of  Static  Scheduling  Research 
Involving  Due-Dates,”  OMEGA.  12:1,  pp.  62-76,  1984. 

[95]  Sethi,  S.  P.,  H.  Yan,  Q.  Zhang,  and  X.  Y.  Zhou.  “Feedback  production  planning  in  a 
stochastic  two  machine  flow  shop:  Asymptotic  analysis  and  computational  results,” 
International  Journal  of  Production  Economics,  30-31,  pp.  79-93,  1993. 

[96]  Sethi,  S.  P.  and  Q.  Zhang.  “Asymptotic  optimality  of  hierarchical  control  in  stochastic 
manufacturing  systems:  A Review,”  NSF  Workshop  on  Hierarchical  Control  for  Real- 
Time  Scheduling  of  Manufacturing  Systems,  Lincoln,  NH,  1992. 


206 


[97]  Sethi,  S.  P.,  Q.  Zhang,  and  X.  Y.  Zhou.  “Hierarchical  Production  Controls  in  a 
Stochastic  Two-machine  Flowshop  with  a Finite  Internal  Buffer,”  Proceedings  of  the 
31st  Conference  on  Decision  and  Control,  Tucson,  AZ,  pp.  2074-2079,  1992. 

[98]  Sethi,  S.  P.,  Q.  Zhang,  and  X.  Y.  Zhou.  “Hierarchical  Control  for  Manufacturing 
Systems  with  Machines  in  Tandem,”  Stochastics  and  Stochastics  Reports,  41,  pp.  89- 
118,  1992. 

[99]  Sethi,  S.  P.  and  X.  Y.  Zhou.  “Feedback  Controls  in  Two-Machine  Flowshops,  Part  I: 
Optimal  Controls  in  the  Deterministic  Case,”  Working  Paper,  Department  of  Man- 
agement, University  of  Toronto,  Canada,  1993. 

[100]  Sethi,  S.  P.  and  X.  Y.  Zhou.  “Feedback  Controls  in  Two-Machine  Flowshops,  Part  II: 
Asymptotic  Optimal  Controls  in  the  Stochastic  Case,”  Working  Paper,  Department 
of  Management,  University  of  Toronto,  Canada,  1993. 

[101]  Sharifnia,  A.,  M.  Caramanis,  and  S.  B.  Gershwin.  “Dynamic  Setup  Scheduling  and 
Flow  Control  in  Manufacturing  Systems,”  Discrete  Event  Dynamic  Systems:  Theory 
and  Applications,  Vol.  1,  pp.  149-175,  1991. 

[102]  Sidney,  J.  B.  “Optimal  Single-Machine  Scheduling  with  Earliness  and  Tardiness 
Penalties,”  Operations  Research,  25,  pp.  62-69,  1977. 

[103]  Srinivatsan,  N.  and  S.  B.  Gershwin.  “Selection  of  Setup  Times  in  a Hierarchically 
Controlled  Manufacturing  System,”  Proceedings  of  the  29th  IEEE  Conference  on 
Decision  and  Control,  Hawaii,  1990. 

[104]  Szwarc,  W.  “Single  Machine  Scheduling  to  Minimize  Absolute  Deviation  of  Comple- 
tion times  From  a Common  Due  Date,”  Naval  Research  Logistics  Quarterly,  36,  pp. 
663-673,  1989. 

[105]  Ullman,  J.  D.  “NP-Complete  scheduling  problems,”  Journal  of  Computation 
System  Sciences,  10,  pp.  384-393,  1975. 

[106]  Ullman,  J.  D.  “Complexity  of  sequencing  problems,”  Computer  and  Job  Shop 
Scheduling  Theory,  E.  G.  Coffman  (ed.),  Wiley,  New  York,  pp.  139-164,  1976. 

[107]  Vani,  V.  and  M.  Raghavachari.  “Deterministic  and  Random  Single  Machine  Schedul- 
ing With  Variance  Minimization,”  Operations  Research,  35,  pp.  111-120,  1987. 

[108]  Wijngaard,  J.  “The  Effect  of  Inter  Stage  Buffer  Storage  on  the  Output  of 
Two  Unreliable  Production  Units  in  Series  With  Different  Production  Rates,” 
A.I.I.E.  Transactions . 11,  pp.  42  ff.,  1979. 

[109]  Yano,  C.  and  Y.  Kim.  “Algorithms  for  a Class  of  Single-Machine  Weighted  Tardiness 
and  Earliness  Problems,”  European  Journal  of  Operational  Research,  51,  pp.  1-12 
1989. 


BIOGRAPHICAL  SKETCH 


Franz-Josef  Kramer  was  born  in  Daun,  Germany,  on  July  8,  1965.  He  received  the 
degree  Diplom-Informatiker  from  the  Department  of  Computer  Science  at  the  University 
of  Kaiserslautern,  Germany,  in  June  1989.  In  July  1989,  he  came  to  the  United  States  to 
pursue  his  studies  in  the  Department  of  Industrial  and  Systems  Engineering.  He  received  his 
Master  of  Science  degree  in  this  department  in  December  1990  and  subsequently  pursued  the 
Ph.D.  degree  with  specialization  in  operations  research.  Throughout  his  graduate  studies 
he  has  been  working  as  a graduate  teaching  and  research  assistant. 


207 


I certify  that  I have  read  this  study  and  that  in  my  opinion  it  conforms  to  accept- 
able standards  of  scholarly  presentation  and  is  fully  adequate,  in  scope  and  quality, 
as  a dissertation  for  the  degree  of  Doctor  of  Philosophy. 

Chung- Yee  Lee,  Chairman 
Associate  Professor  of  Industrial 
and  Systems  Engineering 


I certify  that  I have  read  this  study  and  that  in  my  opinion  it  conforms  to  accept- 
able standards  of  scholarly  presentation  and  is  fully  adequate,  in  scope  and  quality, 
as  a dissertation  for  the  degree  of  Doctor  of  Philosophy. 


Sherman  X.  Bai,  Cochairman 
Assistant  Professor  of  Industrial 
and  Systems  Engineering 


I certify  that  I have  read  this  study  and  that  in  my  opinion  it  conforms  to  accept- 
able standards  of  scholarly  presentation  and  is  fully  adequate,  in  scope  and  quality, 
as  a dissertation  for  the  degree  of  Doctor  of  Philosophy. 


D.  JadK  Elzinga 
Pressor  of  Industrfa 

and  Systems  Engineering 


I certify  that  I have  read  this  study  and  that  in  my  opinion  it  conforms  to  accept- 
able standards  of  scholarly  presentation  and  is  fully  adequate,  in  scope  and  quality, 
as  a dissertation  for  the  degree  of  Doctor  of  Philosophy 


William  Hager 
Professor  of  Mathematics 


I certify  that  I have  read  this  study  and  that  in  my  opinion  it  conforms  to  accept- 
able standards  of  scholarly  presentation  and  is  fully  adequate,  in  scope  and  quality, 
as  a dissertation  for  the  degree  of  Doctor  of  Philosophy. 


Sencer  Yeralan 

Associate  Professor  of  Industrial 
and  Systems  Engineering 


This  dissertation  was  submitted  to  the  Graduate  Faculty  of  the  College  of  En- 
gineering and  to  the  Graduate  School  and  was  accepted  as  partial  fulfillment  of  the 
requirements  for  the  degree  of  Doctor  of  Philosophy. 


August,  1994 


Winfred  M.  Phillips 
Dean,  College  of  Engineering 


Karen  A.  Holbrook 
Dean,  Graduate  School 


